A midsized e-commerce company (~100M annual revenue) faced a lot of challenging questions as they approached 2021:

• What lasting impact will the COVID-19 pandemic have on our business and the market? 
• How can we continue to grow, and how should we set growth goals?
• What is the true impact of our different advertising channels?
• What should our 2021 advertising strategy look like; how do we determine our overall budget, and what is the most effective way to invest that budget? ‍

AdVenture Media’s data analytics team attempted to help provide answers to all of these questions; but along the way, uncovered multiple insights that could reshape the way that we look at this account, and others. Among the takeaways:

• While revenue and ROAS increased during the COVID-19 pandemic, the pandemic itself was not the primary reason for the increase.
• The bulk of the improved performance can instead be attributed to Google\'s automated bidding systems, and other tests conducted throughout 2020.
• YouTube and Google Smart Shopping campaigns proved to drive an incremental lift in overall site revenue, despite not directly being attributed to these individual campaigns within the Google Ads UI.
• A similar phenomenon took place within our Smart Shopping campaigns. In addition to Smart Shopping’s ability to drive direct-response sales, we measured an incremental lift in brand affinity and direct site traffic as a result of our Smart Shopping investment thanks to Smart Shopping’s ability to place targeted ads on multiple placements including Display, YouTube, and Gmail. Similar to our YouTube study, Smart Shopping proved to contribute additional value that was not directly reported in the Google Ads UI.

Platforms Leveraged

• Google Ads
• Facebook Ads
• Google Analytics
• Magento
• Python Programming Tools
• AdVenture Media’s Custom-Built Business Intelligence Software

Background‍

This e-commerce business saw significant growth in 2020. Overall revenue grew 39% YoY, and net profit from their digital ad campaigns increased by 64% YoY. Their primary traffic sources of Google search, Google organic, and Direct increased by 81%, 49%, and 37% respectively. It is natural to assume that the COVID-19 pandemic was the primary cause of these outcomes. But to what extent, and how specifically did it influence shopping behavior? According to Digital Commerce 360, using data from the U.S. Commerce Department, Consumers spent $861.12 billion online with U.S. retailers in 2020, up 44.0% from $598.02 billion in 2019. They conclude that: Changing consumer spending habits as a result of the coronavirus pandemic contributed to the spike in ecommerce sales last year, as statewide lockdowns and fear of contracting the virus kept consumers out of physical stores. This is a theme that many marketers have agreed with. However, there are two issues with this statement: it does not consider additional variables (impact of competition, changing dynamics of advertising platforms, etc.), and it also does not provide guidance about what might happen after vaccines are distributed, stores reopen, and consumers feel comfortable returning to stores. As we look toward 2021 and beyond, we needed a data-driven approach to finding answers to more of these questions.

Measuring a COVID Impact

AdVenture Media’s data science team first sought out to determine how much of an impact, if any, the COVID-19 lockdowns had on the increased performance. According to the aforementioned Digital Commerce 360 analysis: COVID-19-related boosts in online shopping resulted in an additional $174.87 billion in ecommerce revenue in 2020, Digital Commerce 360 estimates. If it weren’t for the bump in online sales from the pandemic, the $861.12 billion in ecommerce sales wouldn’t have been reached until 2022. This estimate is an attempt to provide a macroeconomic snapshot; this does not suggest that every individual online business should have expected a similar percent-lift. Regardless, this estimate was determined by using historic data to predict future outcomes; an analyst would then compare that prediction to actual outcomes. Any gap between the two numbers indicates an impact from an outside variable, such as the 2020 pandemic. Digital Commerce 360’s model predicted that, in normal times, online shopping should have resulted in $687 billion in revenue. The additional $174 billion in actual revenue is being attributed to the pandemic. We developed a similar model for this individual client, using historical data from our ad campaigns as a proxy. Three years’ worth of spend and revenue data were used to predict 2020 revenue. If predicted 2020 revenue was below actual 2020 revenue, we would conclude that an outside variable such as COVID contributed to the lift. In the below chart, the orange bar represents a conservative predicted revenue, while the yellow bar represents an aggressive revenue target. We were astonished to find that actual revenue was significantly lower than multiple versions of the model’s predicted revenue. To the untrained eye, it appears that 2020 was significantly better than previous years. This is true, but only to an extent: it does not consider the dramatic increase in advertising spend, represented by the blue line, that was realized throughout 2020. The increase in revenue is, therefore, more closely tied to the increase in advertising investment than it is to an outside variable such as the pandemic. ‍

Impact of Competition‍

The competitive landscape of 2020 changed dramatically. Lockdowns forced our target audience to rely on online retailers, removing local retail competition. We also saw many competitors reduce their advertising spend, including Amazon.com. While that happened, though, other players entered the market or increased their investments. The following chart outlines our Search Impression Share relative to ad auction competitors throughout 2020. Amazon, highlighted in red, pulled their Google advertising budgets in late March and re-entered the market in mid-May. Another competitor, which shall remain nameless but is highlighted in yellow, followed a similar course of action; however, they did not re-emerge until September. Meanwhile, Wayfair, highlighted in blue, transformed from a non-factor into a primary competitor; seizing the opportunity left from Amazon and other smaller competitors. Finally, Etsy, in green, entered the market to capture a small but significant share. This is further illustrated by tracking our Overlap Rate, highlighting the competitors in which our ads appeared alongside over time: Due to the fact that the Google Ads’ auction exists in a real-time bidding environment, the actual price that advertisers pay is dramatically impacted by the breadth of competition. If fewer competitors existed in the auction, then we should expect to realize a lower CPM. As competitors re-enter the space, CPMs should steadily increase. It would be assumed that overall online competition was lighter during the pandemic, and we should have therefore seen lower CPMs in our account. The opposite was true. On average, CPMs were 12% higher YoY, suggesting that overall online competition was more aggressive in 2020 than in the previous year.

But what about local competition? As mentioned, local lockdowns reduced the amount of in-store retail options available, thus generating a massive influx in demand for e-commerce solutions. If this were true, specifically within this account, we should have realized an increase in our conversion rate. As lockdowns ease, conversion rates should decrease. Again, the opposite was true. The overall account conversion rate in 2020 was 28% lower than 2019; the Search campaign conversion rate decreased by 18%. Our conversion rates were at their lowest point during the earliest days of the pandemic when lockdowns were most strict. We can therefore conclude that local lockdowns did not have a positive impact on our conversion rates throughout 2020. This narrative was further supported by an ROAS Decay Model, which seeks to measure the decrease in expected ROAS for each incremental dollar invested in advertising. In short, the ROAS Decay Model helps illustrate the law of diminishing returns. The ROAS Decay Model proved that it was more difficult to acquire new customers in 2020 than it was in each of the previous five years. It is worth noting that these studies were conducted for just one e-commerce business and are not representative of the larger e-commerce industry. The takeaway, though, is to never assume a causal relationship between any two factors, even if there is a strong anecdotal correlation. It is important to keep digging… and so we did.

So, What Actually Caused The Lift?

We dramatically increased our investment in advertising, year over year, across all channels. The two channels that realized the largest percent increase were Smart Shopping and YouTube. Incremental Lift via YouTube YouTube is a powerful advertising platform that is often misunderstood. The value of YouTube is often not measured within the Google Ads dashboard: it is rare that a user will convert after exclusively seeing a YouTube ad. Instead, the impact is realized over a longer period of time. For example, if two users were to click on a Google Search Ad, but one of them was previously exposed to our brand through a compelling YouTube ad, then we can assume that the predicted conversion rate on that individual is higher than their peer (all else being equal). What’s more, if a user opts to skip a Skippable In-stream YouTube ad, that ad impression will not count as a view, and therefore will not be tracked under any attribution solution. However, a lot of powerful marketing can be done in those first five seconds. Savvy marketers should realize the impact that these free impressions can have on the rest of their campaigns. So while it’s difficult to measure the effectiveness of YouTube ads (and other prospecting efforts, for that matter) directly within the Google Ads’ ROAS column, we can posit that YouTube could be influencing other factors, including direct traffic to the site, branded searches on Google, or other elements of conversion rate and brand affinity. All of these factors ultimately contribute to site revenue, so we used total site revenue as a proxy to help determine the real impact of our YouTube efforts. Using a similar regression model that was used to measure the impact of COVID-19, we sought out to predict our site revenue without any YouTube investment and then compared that to the actual site revenue. Once we had a model that could predict actual site revenue within a small margin of error, we applied that same model to the 2020 numbers. The results proved that YouTube prospecting efforts contributed somewhere between 3.3% and 3.38% of site revenue. That is, if we had not run any YouTube ads, overall site revenue would have been around 3.34% lower than the actual outcome. Incremental Lift via Smart Shopping ‍ Smart Shopping is a campaign type that places product listing ads on multiple placements, including Google Search Results Pages (SERPs), as well as throughout the Google Display Network, YouTube, and Gmail. A common misconception is that most advertisers think of Smart Shopping ads within the confines of the shopping.google.com placement, without realizing that the vast majority of their Smart Shopping impressions are being served on other channels. Our YoY investment in Smart Shopping increased by nearly 40%, and revenue that was attributed to our Smart Shopping campaigns within the Google Ads UI had increased by more than 50%. Once we determined that this significant increase could not just be attributed to the COVID-19 pandemic, we had to evaluate other options to explain the dramatic lift. Our hypothesis was this: if YouTube impressions led to a statistically significant increase in overall site revenue, and if Smart Shopping impressions include YouTube and other similar placements, then it is possible that Smart Shopping is driving incremental revenue that is not measured within the Google Ads UI. This account adopted Smart Shopping campaigns in 2018, and for about a year, ran Standard Shopping campaigns (the legacy shopping campaign type) alongside our new Smart Shopping campaigns. Three unique datasets were available for us to work with: a period where we were exclusively running Standard (legacy) Shopping campaigns, a period where we were running both Standard and Smart Shopping campaigns, and a period where we exclusively ran Smart Shopping campaigns. Therefore, we could measure the impact that these campaigns had compared to one another, in addition to measuring the impact of Smart Shopping as a whole. We used similar methods outlined above, including linear optimization analysis, over a five-year period. We concluded that, for every dollar invested into a Standard Shopping campaign, we produced $5.37 in revenue; for every dollar invested in Smart Shopping, we produced $9.44. This is despite the fact that the Google Ads reported a ROAS between 400% and 500% ROAS for these campaigns over this period. Therefore, not only did we prove that the real value of both campaign types exceed that which was reported in Google Ads, we proved that our Smart Shopping campaigns were 76% more effective at driving incremental site revenue. We conducted an additional analysis to measure any impact of impressions, by themselves, as a means to influence site revenue. For Standard Shopping, no statistically significant correlation existed. That is, any increase in Standard Shopping impressions did not appear to have an impact on site revenue (and instead, revenue from this channel was tied to the actual clicks that were acquired from this channel). However, Smart Shopping impressions did have a statistically significant impact on revenue. That is, an increase in Smart Shopping impressions, holding clicks constant, contributed to an uplift in overall site revenue. Our hypothesis was therefore true, that Smart Shopping impressions (via Display, YouTube, and Gmail) could have a similar brand impact that has been seen with YouTube.‍ Impact of Machine Learning‍ It must be noted that Google’s machine learning and automated systems played a key role in this growth. Smart Shopping in particular is a campaign type that relies almost exclusively on automation and uses machine learning to accurately predict the value of placements throughout the Google Display Network, YouTube, and Gmail. As you scale advertising campaigns, you provide more data to the machine learning algorithms, which increases their ability to predict accurate outcomes. An increase in advertising investment is therefore not just a short-term investment in direct response sales—it’s also an investment in training Google’s systems to make better predictions on your behalf over the long term. ‍

Final Results‍

Overall revenue for 2020 was up more than 70%, compared to 2019. However, we continue to see positive results even throughout 2021 and a gradual re-opening of the U.S. We have been using March 15th, 2020 as the proxy date for the beginning of the COVID-19 pandemic. As of April 13th, 2021, more than a year after that date, we are happy to report that our current revenue numbers continue at a greater than 50% increase YoY. This further supports our hypothesis that the pandemic was not the root cause of our success; instead, this was a result of increased testing and investment throughout our digital advertising channels.   About the author: AdVenture Media is one of New York\'s fastest-growing digital ad agencies, managing over $50 million in media spend.