Advanced Analytics for Marketing Measurement Through Uncertainty during COVID-19

How can marketing leaders uncover the impact of COVID-19 with so many other forces at play? Can we still leverage historical results to predict outcomes given this unprecedented, non-regular event? How will we know the lagging impact of COVID-19 as we shift to stabilization/recovery and revitalization phases?

This article answers these questions and more. While your organization may have access to enormous amounts of data, this unparalleled situation poses unique and complex measurement challenges that require advanced, holistic analytics to answer. This article takes a deeper look at how brands can accurately capture the impact of COVID-19 on business performance even during uncertainty.

This article highlights how brands can accurately capture the impact of COVID-19 on business performance. While many brands have access to big data and analytics, this unparalleled situation poses unique and complex measurement challenges that require advanced analytics to answer.

Read on this article to dive into these questions and more:

• How can we effectively capture the impact of COVID-19 with so many other forces at play?
• Can we still leverage historical results to predict outcomes given this unprecedented, non-regular event?
• How will we know the lagging impact of COVID-19 as we shift to stabilization/recovery and revitalization phases?

Content Summary

Background And Measurement Challenges
Advanced Analytic Framework
Recommended Methodology
Measurement Of COVID-19
Tracking And Monitoring
Summary

Background And Measurement Challenges

COVID-19 has had an unprecedented impact on the world as we know it. No business sector or brand has not been impacted in some way. Brands have had to adjust business and marketing plans and revise performance expectations quickly. Beyond its epidemic impact, there are several other connected forces at play. As we move from crisis to recovery, brands also need to consider consumer mobility and purchase habits, media supply and demand, an economic downturn, and the public health crisis and how all are contributing to changes in the way people are behaving and interacting with brands.

We expect the economic impact of COVID-19 to continue to have an impact well into the future. Recall that it took 4 years for the economy to fully rebound from the Great Recession in 2008, making it imperative for brands not just to think short-term but longer-term as well.

While many brands have access to big data and analytics, this unparalleled situation poses unique measurement challenges:

• How can we effectively capture the impact of COVID-19 with so many other forces at play?
• Can we still leverage historical results to predict outcomes given this unprecedented, non-regular event?
• How will we know the lagging impact of COVID-19 as we shift to stabilization/recovery and revitalization phases?

This article will address these measurement challenges and how Analytic Partners is leveraging our adaptive Commercial Mix Modeling methodology to capture the impact of COVID-19 on business performance accurately.

Advanced Analytic Framework

Recommended Methodology

With so many forces at play, a holistic econometric model is best suited to accurately measure and decompose the impact of COVID-19 and its compounding impact on other business drivers such as media, operations, and direct to consumer marketing.

Below is a simplified formulation of an econometric response model where all controllable and non-controllable drivers are included as predictors (independent variables). The model lends itself to quantification and decomposition of impacts, reporting of core performance metrics (ROI, cost per acquisition, response/unit of support, etc.) and scenario planning (simulation and optimization).

Response=f(Marketing,NonMarketing and Macro Factors)

• Marketing: Paid, owned and earned media (TV, print, radio, OOH, social, search, OLV, OLA, etc.) promotional efforts (events, offers, LTO’s, discounts, free goods, coupons, etc.), CRM (email, direct mail, etc.)
• Non-Marketing: Operational drivers such as store openings, new product innovation, assortment, service lines, out of stocks, pricing, competitive efforts, sales channel effects (incl. channel switching)
• Macro Factors: Weather, seasonality, holidays, macroeconomic effects (housing starts, consumer sentiment, consumer confidence, disposable income, gas prices), consumer trends (e.g. cord-cutting, penetration, diet fads, etc.) government policy changes and others

Holistic models specified as per the above reflect what many may refer to as a ‘marketing mix modelling’ solution. The issue with most marketing mix solutions; however, is that they are too simple and do not accurately reflect the reality of the world today. Analytic Partners takes a proprietary econometric approach that we call Commercial Mix Modeling. Our Commercial Mix Modeling methodology leverages an SEM (structural equation model) framework with a hierarchical (dimensionality) structure. It captures stock up, advertising latency and decay curves, diminishing returns, halo, cannibalization, and differential marketing effectiveness by each component in the hierarchy (e.g. geography, segment, line of business).

One additional differentiating aspect of this adaptive SEM approach is that it enables the measurement indirect pathways across marketing touchpoints: e.g. upper funnel media such as TV impacting lower funnel conversions such as paid search. The model quantifies the amount of influence a marketing channel provides another. Just as with media, conversion pathways are not direct or linear as such our SEM construct captures marketing’s influence on multiple intermediate KPIs on the path to conversion, including brand health. Beyond sales, this approach enables multi-objective (KPI) optimizations.

Our Commercial Mix Modeling approach leverages advanced technology and science to provide a multi-dimensional view of the business with both strategic and tactical insight to reflect the current times. Commercial Mix Modeling goes well beyond simple marketing mix solutions as it is faster, more granular, and can not only tell you what happened but also lend insights into what you need to do next – a critical capability during periods of rapid change. Given the disruptive nature of COVID-19 in the marketplace, it is paramount that it is controlled to ensure a more accurate measurement of other drivers as well as the pandemic’s cascading effects on demand and economic indicators. As brands seek to assess the business impact accurately, it is imperative to capture geographic variations with shelter at home orders. Additionally, as brands adjust to consumer purchase behaviour and route to market – e.g. a likely continued shift to eCommerce – it is critical to understand not just the marketing drivers for each sales channel but also the operational, competitive, and other non-controllable influences. Commercial Mix Modeling provides a deep understanding of the impact by the campaign, geography, store/store type, channel, and customer; enabling brands to adapt and navigate through this dynamically changing environment.

Measurement Of COVID-19

We recommend starting with a holistic measurement framework such as Commercial Mix Modeling that incorporates controllable, non-controllable, and macro-factors to isolate the impact of COVID-19 on the business. From a measurement perspective, there are several factors to consider including time horizon (immediate vs longer-term impact), industry (benefitting or negatively influenced, and to what extent) and unique brand/business dynamics (% of sales impacted, geographic footprint, etc.). As an initial analytical objective, we recommend beginning with descriptive data analysis to help define the impact window in terms of business units, sales channels, and time. The goal is to identify where the impact of COVID-19 may be manifested in the dependent variable and gauge the order of magnitude vs expectation. This helps refine our search for the right data inputs for COVID-19 as businesses are impacted differently.

In the illustration below, there is a clear depression in sales during March 2020, which helps define the COVID-19 impact window. This window will look different by industry and by geography as incidence varied across countries (and even geographies, e.g. Hubei province China, NY state in the US, etc.)

Beyond descriptive analyses, we recommend leveraging several types of data analysis methods, including Inferred Causal Impact through Bayesian Structural Time Series to assess the order of magnitude of the impact. We recommend Bayesian Causal Impact as a best practice where existing measurement is in place. Other approaches where data / prior measurement is limited include Econometric Times Series, simulations, and rule-based approaches. Given we have historical measurement and models in place, we can leverage the Bayesian Causal approach to establish a synthetic, expected baseline. The chart below shows the expectation of sales in the absence of COVID-19 (green dotted line) vs actual (light blue line).

It is key to have a deep understanding of the business model and industry vertical to recognized how COVID-19 has impacted the ability to reach and service customers. With this knowledge, we can align candidate data inputs to be tested empirically in the model. Our fully specified model includes marketing, non-marketing, and known external factors. Still, given the atypical and disruptive nature of COVID-19, we further explore other indicators that may have a significant impact on sales. These factors may provide additional insights into how changes in consumer behaviour: e.g. reduced mobility, increased online shopping impact business performance. Inputs are rigorously tested on significance, independence (multicollinearity), in and out of sample fit (for predictive strength), and data source sustainability. Below is a non-comprehensive example list of approaches and variables tested to capture the impact of COVID-19:

• Bayesian Causal Impact: measures a signal in sales itself and the difference beyond the expected response variable (synthetic baseline)
  • Beyond its usage in the diagnostic phase, the approach produces an additional control input to be integrated into the fully specified model.
• Human Mobility data to account for restricted movement based on government stay-at-home orders
• Macroeconomic Indicators such as consumer sentiment, consumer confidence, etc.
• Financial Indicators such as the VIX (Volatility Index) for financial services firms
• Store Closings and Operational Changes in Services, e.g. no longer offering dining in for restaurants or adding a service, e.g. curbside pick-up, changes in business model B2B to B2C, etc.
• Category Base Sales to capture shifts in consumer demand towards certain product classes, e.g. disinfectants, cleaners, shelf-stable food
• Scaled Indicator Variables to capture Out of Stocks and the impact of stockpiling as a result of the initial panic mode buying at the onset of the pandemic
• COVID-19 Incidence as a leading indicator to government orders to shelter in place etc.
• Google Query Volume for specific search terms such as “lockdown”, “virus” etc.

In the table below, we highlight these candidate variables and how they may be used across different industries. As an example, a Retailer may leverage operational data such as store closures to capture how COVID-19 has impacted their ability to serve customers or leverage licensed human mobility data to capture how movement may have been restricted given government stay-at-home orders.

In our analyses to date, the above have proven useful across a variety of use cases across industries such as QSR, retail, CPG, and financial services.

Tracking And Monitoring

In the current environment, it is not enough to know how much COVID-19 has impacted business. Brands need to know how consumer behaviour has changed and may continue to change, and the impact on marketing and media channels, shopping habits, competitive actions, and overall business performance. The decisions taken in the short term will have ripple effects down the road. With a robust model framework in place that incorporates all business drivers, including the economic and COVID-19 impacts detailed above, brands will have a foundation to monitor business performance on an ongoing basis. As noted above, the current economic climate will influence how consumers react and interact with brands, so it is important to monitor and measure how these changes affect businesses so they can adapt accordingly. As brands move from crisis to stabilization/recovery, it is recommended to continue to track and monitor through:

• Ongoing data feeds that incorporate additional / potentially new data sources such as human mobility data, macroeconomic indicators, etc.
• Frequent measurement through daily/weekly data ingestion and ongoing model updates (‘Live Modeling’)
• Experimentation (A/B testing) to quickly test new strategies, prove out results and enable faster reactions and decision making.
• Continuous Scenario Planning based on most recent data, measurement, experiments, and knowledge available

Frequent data and measurement will enable the agility needed to manage through turbulent times. With ongoing updates, brands can quickly identify any external/macro trends impacting the business, evaluate recent marketing programs, messages, and channel mix and quantify the impact of both controllable and non-controllable factors on business performance. Having these insights on an ongoing basis will ensure brands can sense and respond to evolving consumer needs, adjust quickly, and allocate the right investment accordingly.

A strong foundational measurement framework also provides the ability to experiment through frequent testing. As businesses evolve and adjust to the changing environment, differences in consumer behaviour, and marketplace dynamics, agile learning is imperative. Testing can help test new hypotheses, validate learnings, and inform new strategies. Brands that adopt both holistic, ongoing measurement and agile learning through experimentation will be poised to act quicker and drive competitive advantage. Frequent measurement and testing should also be supported by continual scenario planning.

With a sound, comprehensive model in place, brands can simulate possible outcomes, leveraging best, worst, and mid-case assumptions on non-controllable factors such as the economic impact, competitive reactions, and government regulations. Additionally, brands can understand the impact of spend shifts, reductions, and what is needed to offset declines in both the short and long-term. As models continue to be updated with the latest data, results, and experiments, scenarios can be further refined and optimized for the business objectives.

Summary

COVID-19 has disrupted every business in some capacity, which has influenced business and marketing plans and forecasted performance. In this chaotic state, data and analytics become even more important, and measurement approaches must adapt. While each industry and brand will be impacted differently, a holistic Commercial Mix Modeling measurement framework that includes controllable, non-controllable, macro-factors, and the ability to isolate the impact of COVID-19 is an approach all businesses should adopt. It is critical to update existing models to reflect new consumer behaviour and continuously refresh to assess how these changes impact business performance. Organizations need to continue to scenario plan, refining as latest data and results are available, and test and learn through experimentation to prove out learnings and identify potential risk quickly.

Source: Analytic Partners