Best Practices In Market Mix Modelling
Presenting to you the second part of the “Marketing Measurement” series.
EXECUTIVES UMMA RY
Increased demand by CEOs for greater accountability of marketing spending comes just at a time when proliferating media choices undercut the simplicity of measuring branding or sales impact. Companies can learn valuable lessons from CPG marketers who have begun to untangle multichannel marketing’s effects using sophisticated statistical models. The keys to success lie in scoping the project correctly, gaining the support of stakeholders, and working with a modelling specialist who can apply the findings to business problems
Results of a survey conducted by Forrester and the Association of National Advertisers on marketing accountability revealed substantial interest in market mix modeling as a tool to bring greater accountability to marketing spending.1
LEFT BRAIN PRINCIPLES COME TO MEASUREMENT
Left Brain Marketing — analytical marketing strategies, skills, and processes — is not just for database marketers.2
During the past 15 years, marketers have begun to untangle the sales impact of brand advertising and other elements of an integrated marketing mix, with the help of three technological developments:
1) greater availability of store-level sales data;
2) increasing data processing power; and
3) the application of statistical techniques.3 CPG companies, pulling from the vast data stores of Information Resources, Inc. (IRI) and ACNielsen, have pioneered the technique of modelling marketing tactics with business results. In recent years, this success has begun to interest other industries.
For those not familiar with the concept, a market mix model:
Links marketing spending to sales results
A marketing model lines up a company’s advertising, promotion, and sales activities with product sales over time (see Figure 1). Using statistical techniques such as linear and multivariate regression analysis, statisticians look for correlations between marketing activity and sales volume and then tease out the sales impact of individual media or promotional elements of a campaign. The result is an equation that mathematically describes consumers’ responses to marketing stimuli more comprehensively than a simple direct mail response report and with greater sophistication than a weekly sales report.
Quantifies the impact of multiple demand drivers
Coefficients in the equation describe the relative impact of each marketing activity. But marketing isn’t the only thing that drives demand. For example, the price of gasoline has dampened consumer enthusiasm for SUVs, while a scorching heat wave may spur consumption of soft drinks. Modellers compile data on these external factors, analyze their impact on sales, and include relevant factors as additional coefficients in the model equation.
Reveals marketing’s incremental sales impact
Almost two-thirds of marketers agree on “incremental sales generated by marketing” as the definition of ROI.4 Market mix models typically show that
50% to 60% of sales would likely have occurred in the absence of marketing. The model then assigns the remaining sales to the marketing activity that stimulated them and calculates the sales (or profit) per dollar of spending on each marketing activity.
Analyzes geographic, media, or brand-level budget allocations
Depending on the data involved and the type of analysis applied, market mix modelling comes in three f lavours (see Figure 2). A brand manager can build a marketing mix model comparing the impact of advertising, consumer coupons, and trade promotion, while an advertising manager can create a media mix model of the effects of TV, radio, Internet, and print ads. For the CMO, who must decide how much of the total marketing budget to allocate to each of the company’s brands, a portfolio model can show how to maximize the ROI and can also reveal the halo effect, where advertising for one brand may also drive sales for a related brand.
Models Require Clarity And Commitment
Given the increasing clamour for marketing accountability and the success of market mix models in the CPG industry, it is tempting to view this approach as the answer to all marketing investment questions. Although models are powerful tools, marketers evaluating this technique should understand four limitations. Market mix models:
Statisticians need at least two, and preferably three, years of weekly marketing spending and sales data to build a model. This reliance on the past means that the model can’t project the impact of a medium that hasn’t yet been in the mix and can’t tell if spending has been too low to register an impact. This is common today in online advertising. Also, the model’s accuracy quickly declines when trying to compare the effect of a budget shift of more than 15% with historical trends.5
Start with a business goal
Market mix modelling is an umbrella term encompassing a range of techniques that can answer a variety of business questions. Marketers must clearly articulate the marketing decisions they are trying to support and the hypotheses they want to test to enable the modeller to bring the right data and analytic techniques to bear. But relating the model to important business issues helps to win approval for the project.
Require a significant commitment
Marketers can expect to spend at least $50,000 to $100,000 to hire a marketing modelling expert, as well as additional costs if they purchase external data. As models become a routine part of marketing, companies will also need to put effort into integration projects to automatically pull data that is buried in the data warehouse or internal finance and CRM systems. Incorporating usage of the models throughout the marketing process requires further change management.
Aren’t right for all industries
Market mix models are most appropriate for brands that have large budgets, a highly diverse marketing mix, and distribution via indirect channels. B2B companies whose marketing objective is to drive leads and spend little on corporate branding are better served by improving their lead management systems.6 Several vendors report that models usually result in a 10% gain in marketing efficiency; thus, smallbudget brands, where these savings won’t offset the model costs, shouldn’t use the technique. Similarly, catalogue retailers with little spending outside of direct mail won’t derive as much benefit. Products like movies, where each release is a new product launch, won’t have the historical data to build the model.
THE MARKET MIX MODELING VOYAGE
Given the effort and expense involved, companies should carefully plan and execute a new modelling initiative. As one retailer noted, “You have to approach a model like a real IT project.” With a decade and a half of history behind this technique, marketers can follow industry best practices but should be ready to tailor the approach to their specific industry, business problems, and data availability. These projects evolve in three phases (see Figure 3):
1. Plan: Scoping the project and selecting a modelling partner.
2. Prepare: Refining the scope and gathering required data.
3. Build: Creating and refining the model equation, implementing model findings, comparing them with actual results, and enhancing the model’s predictive ability.
Phase 1: Plan The Project And Pick The Right Vendor
While companies with econometric modelling staff may be tempted to go it alone, nuances of marketing modelling require the expertise of a specialist. In addition to marketing knowledge,
modelling partners should have the f lexibility to adapt their approach, the ability to clearly explain the model’s inner workings, and the willingness to help the company access and use the model on a routine basis. Companies should evaluate a modeller’s:
Skills beyond statistical expertise
Banks, auto companies, and some retailers already do statistical modelling of the economy to shape business planning. While marketing modelling uses many of the same statistical techniques, the similarity ends there. The greatest difference is in adjusting marketing spending to account for the lingering effect that an ad campaign has, even after it is over. Transforming marketing spending data into “ad stock” requires knowledge that an econometrician doesn’t have of the decay rate of the impact of an ad campaign.7 Modellers also need to understand common media concepts like gross rating points (GRP) and circulation and must be accustomed to incorporating these disparate data types into a model.
Models for financial services, retail, and pharmaceutical companies will encounter different issues with distribution channels, marketing mix, and data availability. But just because modellers don’t have relevant industry experience isn’t a reason to dismiss them. To create its shortlist of potential marketing partners, Wachovia issued a request for information (RFI) describing the business problem and asking vendors to discuss the approach they would take. The responses allowed Wachovia to determine which vendors had the greatest f lexibility and creativity to adapt their experience to the bank’s needs.
Some modellers regard the model as their intellectual property and only give the client reports of results. But to get the most out of the modelling investment, marketers should make it clear that over time, they will need increasing access to the model to update it and use it on a routine basis for planning, strategy, and ongoing evaluation of marketing effectiveness.
Forrester has heard repeatedly that one of the hardest parts of the model building process is solving the technical and organizational hurdles to accessing necessary data from internal systems and databases. As part of the project framework, companies should plan to build automated data pulls to ease subsequent model updates. Modellers should be able to provide examples of similar projects for other clients and demonstrate the technical skills to define and direct the work.
Phase 2: Prepare The Data And Organization
With a modeller onboard, the next task is to carefully plan the project, identify data sources, and engage other departments or stakeholders that will be affected by the project and the model’s recommendations. This preparation is critical to success, and marketers should guide the project through a series of meetings to build a consensus around the desired outcome from the model and gain support for their continued participation.
Refine the scope
Using a model to determine marketing ROI sounds deceptively simple, but before modelling can begin, the company needs to move from the high-level project objectives to a more detailed definition of the model: which brands will be involved, what the metric of success will be, what type of modelling techniques are best, etc. These decisions can dramatically affect the time, cost, and resources required for the model. One retailer we spoke with spent the first 60 days working with the vendor and internal groups to clarify such issues as whether to use increased sales or profits as the metric and whether to look at short-, medium-, or long-term effects.
Gather the data
The type of model and the metric will begin to shape the data requirements. Planning should begin to identify which internal and external data sources will need to be tapped. It is not unusual to begin with 20 or more data sources, each of which may require multiple rounds to prepare the data correctly. Companies should staff adequately for this task. One financial services company we spoke with had assigned two employees full-time for two months to seek out, extract, and prepare necessary data from internal systems.
Phase 3: Build, Validate, And Enhance The Model
Building a model is a lot of science, but it is also partly an art, requiring several rounds of evaluation and refinement before accepting a finished product. Even so, sceptics are likely to reject the model until a campaign based on its recommendations runs in the market and its projections come true.
Once a model proves accurate, marketers should work to expand its power and ability to guide business decisions. The following steps should be taken:
Iterate the model
Initial drafts of the model should be scrutinized for two factors. First, is the “R-squared” (the closeness of the correlation between a variable and sales) high enough?
If not, the model may need added variables, more detailed data, or, in extreme cases, it may be abandoned. Second, decide if the correlations identified by the model are coincidental or causal. Stakeholders from areas such as finance, sales, and operations should continue their involvement through these stages to help marketing identify spurious correlations. Beyond improving the better model, this will minimize the potential for groups that don’t like the model, on the grounds that a “black box” formula doesn’t accurately ref lect the dynamics of the business.
Validate the forecasts
Models typically identify significant changes in the media used or the budget allocated to different campaign elements. None of the companies we spoke with automatically shifted their plans to exactly mirror the model’s recommendations. Instead, they said they would make a less drastic, but still measurable, change in their plans, run a campaign, and match up marketplace results to the model’s forecasts. Only when real results prove the model’s predictive accuracy do companies begin to use the model to drive strategic marketing decisions.
Construct tests to expand the model’s power
Because models emerge from historical data, they highlight what worked best among past campaigns. But new media forms, such as product placements, blogs, or, in many cases, online advertising, may not have accumulated enough data for the model to ref lect them. To expand the model’s sensitivity, companies should construct a number of different marketing plans in sets of test markets, track the results, and then use the model to add new variables and adjust the coefficient of previously known variables.
CHOOSING THE RIGHT MARKET MIX MODEL PARTNER
In the early days, the modelling industry was made up of a handful of small, independent boutiques. But major agency holding companies, technology consultants, and software companies see business opportunities in marketers’ measurement pain (see Figure 4, see Figure 5, and see Figure 6). Not only does this broader marketplace give marketers more choices, but it also gives them the ability to assemble their own best-of-breed measurement systems by picking the top modeller for their needs, embedding that model in a technology platform, and then engaging the agency or a consultant to implement the changes needed to enhance their business.
Specialist boutiques thrive
Since the early ’90s, the power of PCs and software has enabled anyone with a strong background in econometrics and statistics to hang out a shingle as a marketing modeller. Early entrants like Marketing Analytics continue to attract blue-chip clients like Kraft Foods and Nestlé, while newer entrants like iKnowtion continue to spring up. A full-service approach and deep experience make them good choices for companies beginning the modelling learning curve. However, companies need to make sure that these firms offer complete access to the model, in the event that they decide to bring it in-house or use it in conjunction with another application.
Marketing services companies expand offerings
Ad agencies like DDB Worldwide Communications and database marketers like Target base have traditionally served clients by creating and implementing marketing campaigns. As clients have begun to demand better measurement of these efforts, these firms have added modelling expertise to their lists of services. Because they are marketing companies at heart, these types of vendors are often in the best position to enhance the model with deeper media or consumer data. Make sure there is a strong separation of responsibility between the implementation and the measurement sides of the business to avoid biasing the analysis.
Software companies offer a measurement platform
The combination of data storage needs, analytic tools, and processing capacity that drives modelling makes it inevitable that a company like Veridiem would enter the field with a platform application. As companies embrace modelling as a core aspect of their marketing planning, a platform like Veridiem’s gives marketers greater access to the model to conduct what-if analyses of different marketing plan options, while simultaneously grabbing new marketing data to enable more timely model updates. But a platform alone won’t improve marketing; companies must substantially change marketing processes to get value out of this approach.
Hybrids combine modeling, technology, and consulting
As the ecosystem of tools and services becomes more complex, vendors have tried to simplify the planning and management of models with more comprehensive solutions. Marketing Management Analytics (MMA), one of the original boutique modellers, launched its Avista ASP service in March 2005, i n response to clients’ requests for better access, faster updates, and better tools to get value from the models. Accenture provides the full range of strategic consulting, technology building, and change management services. Firms migrating to this class of vendor should make sure that t hey are not restricted to using only this f ir m’s models but can import and continue to use their existing models.
WHAT IT MEANS
MODELS WILL SUBSUME THE MARKETING PROCESS
For the CEO, measurement is as much about predictability for Wall Street as it is about improving the efficiency of the marketing budget. New types of models and new technologies that make them available to every marketing manager will ensure that companies rely more and more on their models to shape marketing plans that even an investor could love.
Models will evolve from results analysis to planning tools
Marketing models have become central to marketing and advertising decisions at leading consumer goods companies including Kraft, General Mills, and Coca-Cola. As these companies have gained experience over time, they have begun to run what-if scenarios at different budget levels and media allocations to support annual marketing plan development. Today’s Microsoft
Excel simulator tools will yield to more sophisticated systems to track, collect, and model marketing in real time.
Data will move from aggregate sales to individual consumer behaviour
In CPG, the most important data comes from ACNielsen or IRI syndicated sales databases. As Left Brain Marketing expands the number of companies with individual consumer data enriched with behavioural insight like site visits and marketing preferences, market mix models will incorporate more granular data. Traditional database hosting firms like Epsilon and Acxiom
will follow Target base’s lead, adding statisticians to build consumersegment models, drawing on individual transaction-level data that the firms hold for their clients.
Marketing operations will migrate toward finance
As models enable more predictable results from marketing, CFOs will see the modelling process as an extension of the financial controls that they wield in other parts of the organization. To prevent a total takeover of marketing by the finance department, CMOs will instil greater discipline in executing and tracking marketing program results. Creative marketers who chafe under models’ restrictions will redefine their jobs away from managing ad campaigns to new product development, positioning, and communication strategies.
Market research will become less quantitative
As finance exerts greater control in day-to-day execution, CMOs will bolster marketing’s inf luence by rebuilding marketing research’s capabilities. Their new mission: Find the unique consumer insights that can drive innovative new products. Market research will delve more into “soft” research techniques like anthropological studies and monitoring online consumer-generated media to divine consumers’ latent wants, needs, and motivations.
NOTES & RESOURCES
Forrester interviewed 31 vendor and user companies, including: Accenture, ACNielsen, The Advertising Research Foundation, Dratfield Analytics, iKnowtion, Information Resources Inc., Knowledge Networks, Sequent Partners, and Veridiem.Related Research Documents
“What B2B Marketers Need From Technology” April 8, 2005, Trends “Where Is Marketing Measurement Headed?” January 13, 2005, Trends “Left Brain Marketing” April 6, 2004, Forrester Big Idea