Marketers and media planners need to reassess their assumptions about the importance of programming environment to advertising engagement
At a time when advertising exposures occur in increasingly diverse environments on a wide range of technologies, the audience metrics that once mattered most now seem incomplete. And when the audiences for some ads—just like the products they advertise—are becoming narrower, the need to make a deeper impression on a smaller group of people may trump the conventional concentration
on reach.
Meanwhile, marketers are being challenged more than ever to demonstrate the return on their investments. For advertising in both traditional and new media, they are finding that the metrics have not nearly kept up with the times; the difficulty is not just finding a robust (and consumer-respectful) methodology to measure exposure to a commercial that has been TiVo'd or an ad seen on a cell phone, but also finding a measurement that is relevant, accurate,
and reliable.
One index that has risen to dominate discussions of media and ad effectiveness is engagement. This goes beyond a measurement of "time spent" to look at the "quality" of an ad exposure. With the ad viewed via TiVo, for example, it might take into account whether the ad had a clearly visible logo, whether it was viewed in fast-forward mode, and where it landed in the commercial pod. For cell phone advertising, engagement measures might look at where the usage session occurred and whether it was primarily for information or entertainment.
Engagement speaks directly to the need to make deeper impressions—not just gross impressions, and how best that might be done. And it has captured the attention of the advertising and media industries; the Advertising Research Foundation (ARF) has made a much-promoted attempt to define engagement, and Nielsen Media Research has announced its intention to address engagement in the context of television.
Is media engagement undervalued?
Media planners tend to talk separately about media engagement and advertising engagement. Some argue that the two experiences are linked; others suggest that they operate independently. Typically, media engagement is judged to be the less important of the two. The hypothesis that media engagement carries over to the advertising has not been widely embraced; ultimately it is exposure to advertising that interests the marketers who pay for it. (Ironically, TV ratings for commercials are not in general use today, when the need to supplement commercial ratings with other metrics is now clearly at hand.)
But the theory that media and advertising engagement are connected is worth exploring in a systematic way. If one could demonstrate (at least for certain environments) that there is a link between the two—that engagement with the medium leads to engagement with the advertising—then measures of media engagement could play an important role in enhancing the media-planning process. Imagine the value of knowing which networks or programs are more likely to deliver an audience receptive to advertising.
For the better part of fifteen years, Knowledge Networks/Statistical Research (KN/SRI) has been studying what it means to be "in" the television audience. KN/SRI has approached this question from both attitudinal and behavioral perspectives, and KN/SRI's research points toward the possibility of a link between media engagement and advertising engagement. Certain mindsets and usage patterns correlate positively with conditions often associated with advertising receptivity—for example, "eyes-on" involvement with television and less overall channel switching. Given the industry's current interest in engagement (e.g., networks' guaranteeing their delivery of advertising messages and the introduction of a host of new metrics serving as surrogates for the value of engagement), the time seemed right to test an innovative approach to measuring the association between media and advertising engagement.
To conduct a carefully controlled advertising research experiment, it is essential to set boundaries. In the context of today's television world, this means carving out a well-defined territory. FOX News Channel (FNC) has worked closely with KN/SRI for over five years to understand the relationships 24-hour cable news networks have with their viewers, and how these television viewers engage with primetime content. In sum, there was a natural fit of topic and companies.
Together, KN/SRI and FNC set their sights on studying engagement in the context of primetime for the twenty-four hour cable news network. The goal was to design a tightly structured experiment that measured behavior in as natural a setting as possible; we wanted to test in real viewing situations with real advertising. Recruit-to-view samples and specially prepared advertising reels were not options for us.
Media engagement is usually considered less important than advertising engagement |
Defining standards for media "wantedness"
Our hypothesis was simple: Engagement with the medium leads to engagement with the advertising. Our operational definition of media engagement involved measures of brand resonance, "wantedness," and "involvement"(i.e., attentive viewing/listening minutes), while our working definition of advertising engagement was recall.
The first part of the hypothesis called for a measure of media engagement. Using the Knowledge Networks RDD-based online panel1, we surveyed a random sample of more than 3,000 adults in October 2005 about their news habits, viewing of twenty-four hour cable news networks, and "engagement" with three networks—CNN, FNC, and MSNBC. To measure the relationship each network has with its viewers, we used Professor Kevin Lane Keller's Consumer-Based Brand Equity model of brand resonance. Brand resonance refers to the nature of the relationship that customers have with the brand and the extent to which customers feel they are "in synch" with the brand. It can be broken down into four categories:
- behavioral loyalty
- attitudinal attachment
- sense of community
- active engagement
To measure media "wantedness," we used a KN/SRI-developed model built around the concepts of value, effort, necessity, and persistence. And, using "yesterday's" primetime viewing behavior, we calculated for each of the three networks the proportion of total tuning minutes in conditions associated with the highest involvement.2
The FOX News Channel ranked first in all three measures of media engagement (see Figures 1 & 2), with areas of greatest difference including
- fostering a sense of community among viewers;
- prompting conversation—i.e., viewers enjoy telling others about the network;
- being irreplaceable;
- viewers would watch less TV if this network went away;
- would go out of their way to record favorite programs on this network if they knew they might miss them;
- would pay extra to be sure to have continued access to this network;
- habitual, or planned, viewing.
Based on the media engagement results (i.e., FNC delivers the most engaged audience, compared with CNN and MSNBC), our hypothesis would predict that the FOX News Channel would have the highest level of advertising recall. To test this hypothesis, KN/SRI conducted a second interview among all the preidentified regular twenty-four-hour cable news viewers of CNN, FNC, or MSNBC. This second interview was fielded approximately one week after the first interview, also using the Knowledge Networks online panel.
Among viewers of the three cable news networks, FOX News Channel viewers had the greatest opportunity to see commercials |
Unaided recall as an indicator of engagement
KN/SRI asked all regular twenty-four-hour cable news channel viewers to complete a survey about their evening activities before they went to bed; care was taken not to mention television, cable news networks, etc. Respondents were screened first for any primetime television viewing, second for twenty-four-hour cable news channel viewing, and third for viewing of the three subject networks—, FNC, and MSNBC. Viewers of CNN, FNC, and/or MSNBC qualified for the second interview. (see Chart 1)
For each of the three networks viewed during primetime "earlier tonight," respondents were asked a series of questions about how they viewed—length of time, audience composition, channel changing, etc .— and whether they recalled seeing any commercials while they were watching. If the answer to the commercial recall question was "yes," KN/SRI collected unaided descriptions of the commercials, including the names of companies, brands, products, services, and/or any descriptions of the commercial. Because the survey became "available" to respondents at 9PM local time and was "shut down" at 3AM local time, the recall period was effectively limited to a maximum of seven hours (8PM to 3AM).
The methodological decision to rely on unaided advertising recall was purposeful; simply stated, it was the strongest possible indicator of television advertising engagement in KN/SRI's technical judgment. But to be sure that the viewers' recall was accurate, all responses were "matched" against programming on all three networks, adjusted for time zones. Further, KN/SRI coded each response for its "degree" of recall. Because it was impossible to capture all local advertising "on tape," the accreditation of local ad recall was more lenient. However, local advertising accounted for less than two percent of all ad recall. Finally, for each recalled commercial, respondents also were asked to evaluate how much attention they paid to the ad compared with the program that carried the ad.
Media Engagement "Tonight." Consistent with the results from the first interview, the FOX News Channel again attracted the most engaged viewers—more likely to make a special effort to watch, longer tuning duration, fewer channel changes, and more attentive viewing. In short, among viewers of the three networks, FNC viewers had the greatest opportunity to see commercials. But, did they remember what they saw?
Advertising Recall "Tonight." Self-reports of primetime advertising recall—respondents saying that they recalled any advertising—were highest among FNC viewers (38 percent). When all recalled descriptions were checked against the videotapes, "validated" recall levels dropped considerably, as one might expect in an experiment of this nature, but FNC remained on top with 16 percent of its viewers correctly recalling at least one ad from "tonight's" viewing. (see Figure 3)
And, on average, FNC viewers recalled more primetime commercials—2.2 versus 1.9 for MSNBC and 1.7 for CNN. Because FOX News Channel viewers were tuned to that network for a longer period of time—longer than CNN viewers tuned to CNN or MSNBC viewers tuned to MSNBC—one could argue that they have a greater chance for commercial recall (stemming from more opportunities for exposure). Nonetheless, when we control for time spent viewing, one still finds FNC with the highest number of ads recalled per thirty minutes of viewing.
Degree of Recall. Each "validated" commercial description was coded on a scale of one to four for the degree of information it contained:
- General description that matched a commercial shown, but no brand mention
- Detailed description that matched a commercial shown, but no/incorrect brand mention
- General description that matched a commercial shown, including correct brand mention
- Detailed description that matched a commercial shown, including correct brand mention
Averaging scores across all "validated" commercials shows similar levels of recalled detail for FNC (3.4) and MSNBC (3.4), followed by CNN (3.0). On balance, MSNBC produced the highest proportion of "fours"—detailed descriptions with correct brand identification—and FNC produced the lowest proportion of "ones"—general descriptions with no brand identification. (see Figure 4)
Attention to Commercials. Recent musings about a shift from metrics that measure how many could have seen an ad to how many people actually paid attention to an ad prompted us to include an attention question for each recalled commercial. Specifically, we asked respondents whether they watched each recalled commercial "as attentively" as they watched the [network] programming, "somewhat less attentively" than they watched the [network] programming, or "much less attentively" than they watched the [network] programming. The option of "more attentively" was not considered a realistic outcome and was not offered.
Across all three networks, commercials were viewed with less attention than the programming that carried them. The ads recalled on MSNBC were reported to have been watched more attentively than either the ads recalled on CNN or FNC. But the balance of number of ads recalled with attentiveness still supports FNC's overall effectiveness.
Placing your bets where engagement is highest
One of the first lessons that is taught to the aspiring statistical researcher is that correlation is not causality. Across all three twenty-four-hour cable news networks, the viewers who correctly recalled advertising posted higher media engagement scores than the viewers who did not recall any advertising. Does this mean that media engagement will lead to advertising engagement? The answer is no. But it might be an indication that an advertiser could see a better return on its investment if the television advertising investment is placed in programming known to involve its viewers.
Until the media and marketing research industries derive the elusive, all-purpose advertising engagement metric, gauging the networks and programming that cultivate engagement may provide a leading indicator—one with vast implications for planning. This innovative media and advertising engagement study represents a principled and practical starting effort, one that could well be applied in other situations and settings, including different types of TV programming, and even different media. If media and advertising engagement are to be added to marketing science's ROI calculations, more common-sense thinking and statistical research will need to be done. In the end, where the measurement of engagement is concerned, the industry might do well to embrace the evocative words of T.S. Eliot:
We shall not cease from exploration
And the end of all our exploring
Will be to arrive where we started
And know the place for the first time.
—T.S. Eliot, "Four Quartets," Little Gidding, Section V
Maura Clancey is Vice President and Managing Director for Knowledge Networks/SRI. She has been testing innovative forms of media measurement for the past 16 years. She can be reached at mclancey@knowledgenetworks.com.
Paul Rittenberg is Senior Vice President of Advertising Sales and Market Research for the FOX News Channel. He is responsible for overseeing the entire FOX News advertising sales operation. He can be reached at Paul.Rittenberg@FOXNews.com.
Footnotes:
1 Knowledge Networks maintains the only online consumer panel based on a representative sample of the U.S. population.
2 Based on fifteen years of HPU research, KN/SRI identified several "conditions" that correlate positively with involvement with television programming, such as purposeful viewing, planned viewing, and the absence of other activities while watching. We added the proportion of tuning minutes in each of the seven conditions and divided by seven. This calculation gives equal weight to each of the seven conditions. There is no accounting for interaction effects.
