In September for our Data-Driven Digital Community webinar, we talked to James Nicholson, Digital Technology Consultant about how the COVID-19 pandemic has pushed the world into uncharted waters, particularly when it comes to predicting how people will behave in this environment, let alone dispatching meaningful and effective actions at scale.
In this webinar, we cover how organisations who bridge the gap between their first-party data and their marketing and business operations automation have flourished during these trying times.
Watch the webinar below. TL;DR? View the slides and read the recap.
The Great Divide: Insight to Action
I think it’s fair to say that we live in unprecedented times, both professionally and as human beings. And, although it might be hard to believe…
I am not just referring to the radical changes that the COVID-19 pandemic has imposed on us.
Many of the changes that we are living through right now have been quite some time in the making.
It just so happens that all these changes have culminated in the ultimate ‘rip the bandaid off’, ‘brace for impact’, ‘ready or not here I come’ moment; thanks for that 2020.
To shake things up a little, let’s take a look at some of the changes outside that of COVID-19. I’m sure we have all spent so much time talking about the Coronavirus that exploring some of the other changes we’re facing together will seem almost like a holiday!
So, with that, let’s dive in.
Privacy
The ‘P’ Word
I hope you’re sitting down because we’re going to talk about the P-word. Privacy: the very word makes even the most stoic amongst us develop a nervous twitch, and for good reason.
Over the last 3 years alone, we have seen some seriously game-changing legislation take effect across Europe and the Americas, including the GDPR, the California Consumer Privacy Act, and the LGPD.
If you were anything like me prior to the roll-out of the GDPR, some serious time went into figuring out what information you can and can’t collect, the steps you need to take before you can collect this data, which vendors and teams have access to this data in our collection and reporting pipeline, and so on.
I was burning more calories trying to figure out which of my client’s were data ‘processors’ vs ‘controllers’ than I was at the gym.
Data Privacy in Australia
And then there is Australia, who — compared to many other countries — has been quite slow to act with regards to privacy on the web.
Don’t be mistaken though, there are some pretty interesting developments in the works, with Australian regulators (like the ACCC) actively questioning the (often opaque) audience profiling capabilities exposed by ad tech vendors such as Facebook, Google Ads, and LinkedIn.
Needless to say, watch this space.
Governments are not alone in their concern for their subjects privacy online, however. Personal privacy technology is a rapidly growing industry with the appetite for maintaining privacy on the web is becoming more pervasive than ever before for the everyday consumer.
And this is hardly surprising. It feels like we were hearing of a new data breach on an almost daily basis prior to the COVID-19 pandemic. And while the news of these breaches has slowed a little (particularly since the news cycle has been almost exclusively occupied by pandemic news), the breaches have not.
For instance, last May the Australian graphic design tool website Canva suffered an attack that exposed the email addresses, full names, and home addresses of some 139 million users.
In September of the same year, several unprotected databases were discovered that contained the phone numbers (and, in some cases, age and gender) for almost 20% of Facebook’s 2.3 billion users — that is almost 420 million records exposed to the public web.
Now, if you are feeling an urge to panic buy tin-foil for your anti-Illuminati headwear, you are not alone.
While the figures on tin-foil hat usage are still unknown, we do know that many Australians are taking their privacy into their own hands, with one in four Aussies browsing the web with an Ad and Tracker blocking services like Ghostery, and over a quarter using a service like Tor or a VPN to protect their privacy online.
We’ve come a long way with respect to our privacy infrastructure, there is no arguing that, but we clearly have a long road ahead of us. For many people, privacy is something that is often quite complicated, and the barrier to entry is high as a result.
For somebody like my dad, it’s a case of crossing your fingers, throwing salt over your left shoulder, and hoping for the best.
And it’s precisely this barrier that is fueling one of the biggest privacy changes that we have ever seen. So what is this change and where is it happening? Chances are the answer is right in front of your nose — literally!
Data Privacy & Browsers
Browser vendors are baking in many of the privacy mechanisms that may be beyond the technological or financial reach of some users. This is quite a paradigm shift in many ways, as the global consensus is that privacy on the web is a right that should be enjoyed by all, irrespective of the protections put in place by a government, or the accessibility of tools like a VPN.
Browser vendors like Brave and the self-describing Epic Privacy Browser (who actively block trackers and even tag deployment systems like Google Tag Manager) are showing the appeal of privacy by default.
And this is placing some serious pressure on big players in the browser market like Safari, Chrome, and Samsung Internet, who are beginning to introduce features like Intelligent Tracking prevention, limited access to browser APIs – and, in some cases, only allowing access to after a user has granted explicit permission to do so. Then there is DNS over HTTPS (in the case of Firefox) and – here it comes …
Blocking Third-Party Cookies
blocking third-party cookies. You knew I’d mention this eventually…
Now, I promise that I didn’t bring all this up to revive the nervous twitch you thought you finally got rid of.
I mention privacy because it is only now beginning to seriously impact the technologies many of us use on a daily basis such as Google Analytics, Adobe Analytics, for reporting and Floodlight tags, Facebook Pixel, and LinkedIn Insights tag just to name a few.
For my own clients, I would be seeing somewhere in the range of 15% to 20% data loss due to ad blockers and enhanced browser privacy features alone (when comparing to server logs, Google and Bing Webmaster Tools, and first-party tracking solutions that I have developed).
Less Data
I am also seeing first hand a degradation in the level of detail being collected across a suite of web analytics platforms, like browser version, operating system version, and even how referral source is attributed (in some instances), and this is in no small part due to downgraded access to browser APIs that once yielded high definition data.
I am sure many of you have seen an increase of ‘not set’ in your Google Analytics reports and ‘Unspecified’ or ‘Unknown’ in Adobe Analytics. The fact that these blind spots are beginning to manifest themselves in our reports at what might be considered the ‘dawn’ of mass internet security and privacy is a pretty scary prospect for many die-hard web analytics professionals.
For me, seeing these ‘holes’ really brought home the idea that all this talk of legislation reform, privacy feature releases, and use of privacy tools wasn’t just an interesting conversation point or academic exercise – it’s real life, and we are seeing the effects of this first hand.
Marketing Automation
It’s not just reporting that is taking a hit, either. Marketing automation and audience creation are beginning to become more challenging, and in some cases, less effective. Many marketing technologies heavily rely on browser storage (like cookies or the localStorage API) to persist information about a user between pages and sessions, and when you take this persistence away (or limit its lifespan) your audience composition can appear to change.
Experiment
As a little thought experiment, let’s say that you and your team have been running a comprehensive media campaign to acquire new subscription customers and to supplement this, you have some search campaigns up and running.
You don’t want to be as aggressive in ad auctions when one of your customers is searching as they may just be wanting to pay a bill or manage their account, and so you increase your bids for new users. This allows you to maximise your visibility for your target user type.
Naturally, things are going swimmingly, because you and your team are the avengers of digital marketing.
But suddenly, new customer acquisition volumes drop despite there being no changes to your marketing activity and no external newsworthy factors that seem to be influencing your prospective users.
Changes in Data Collection
This is exactly what happened for many advertisers when version 2.2 of Safari’s Intelligent Tracking Prevention was released last June; many advertisers and publishers saw their web analytics platforms reporting an increase in new users, to the tune of 8% in some cases.
Now, as humans (and data nerds), we know that the increase of new users in this scenario is due to a change in how the data is collected and not subjects in the collection pool. Automation models, on the other hand, rely heavily on human intervention to prevent these kinds of undesirable side effects from emerging, and sometimes even this is not enough to restore order.
Now, I appreciate that this sounds all rather ‘doomsday-esque’, but there are some pretty cool things you can do to mitigate the effects here and use these quirks to give your reporting and automation an edge.
I will explain what I mean about this in a moment, but first, just in case you forgot …
The Year of COVID-19 and Lockdowns
I’m sorry, just when you were getting comfortable with the idea that all your personal information is probably floating around in the public domain for all to see, I have to ruin the moment and remind you that we are in the thick of a global pandemic.
Let’s break this down.
So, we know that data collection is changing, and this change is beginning to influence how our marketing automation works for our organisations, how we report and evaluate performance, and ultimately impacts the decisions we make.
And to add an extra layer of complexity to an already mind-boggling situation we have this radical shift in human behaviour, not just online but almost every facet of human existence, brought on by the COVID-19 pandemic.
Between mid-March and early April, state and federal governments had decided that strict social isolation measures should take effect, and many of us transitioned to working from home. Over this time, some very interesting things have happened.
Change in Internet Traffic
Australia saw some of the fastest growth in internet usage, increasing by close to 50% compared to the beginning of the year, with much of the burden moving away from the well-resourced commercial infrastructure and being put on consumer internet connections.
By and large, our internet infrastructure did not fare well, with network congestion reaching (for some) unworkable levels, particularly in Canberra, Melbourne, and Brisbane.
5.2M Aussie Households Shopped Online in April 2020
Now, despite the trying network conditions, Australian’s were not deterred from engaging in online activities, many for the first time, with over 5 million households shopping online in April alone, and 200 thousand of these for the very first time.
It’s hard to properly comprehend the scale of this growth in online shoppers, so to put this into perspective the four days from Easter Tuesday 2020 was 2% higher when compared to the four days from Cyber Monday in 2019, which is pretty staggering.
+40% of Consumers Globally Will Continue Post-crisis
This appetite for online shopping is not likely to go anywhere anytime soon, with a significant number of consumers expressing their preference to shop online even after the COVID-19 crisis.
Mobile vs. Desktop
And as if this wasn’t crazy enough, there has been a resurgence in large-screened devices with Australian desktop device usership increasing by 12.07%, while mobile device usage shrunk 13.44%, changing the content consumption landscape in ways that are difficult to predict.
Noise in the Data
While we are seeing some large scale trends, the thing that I am finding the most interesting (and also the most challenging) is the noise in so much of the data that I am reviewing. Indeed, even doing research for this talk was pretty challenging, as much of the data and reports had conflicting information.
The rapidly changing environment that people are living in right now – people going in and out of lockdown, consumer confidence growing and shrinking based on the availability of government financial support, players dropping out of the market (think Tiger Air) and others participating for the very first time — is creating some really interesting challenges, particularly for marketing automation.
Google’s Marketing Automation
Earlier we touched on the fact that data obfuscation from privacy features in browsers can impact the effectiveness of marketing automation and audience creation.
But what’s really going on under the hood?
For a moment, let’s zoom in on the Google Ads automation suite – specifically, what Google calls ‘Smart Bidding’. As a quick refresher on how Google’s search ad auction works for those who actually have a social life and don’t spend their entire life working with or reading about marketing automation…
When a user does a search on Google, Google will take the query and process it, passing it to the Google Ads service to see if the intent matches any of the criteria that advertisers may be wanting to target. From there, an ad auction is run (in real-time) and the participating advertisers submit their bids. These bids influence which advertisers will serve ads to the user, and in what order, and so it’s important that your bid reflects the expected value a user exposed to your ad may yield.
Smart Bidding
This is where Google’s Smart Bidding comes in. Smart Bidding describes a subset of bid-automation strategies in Google Ads that works by accessing a bunch of different signals from different sources (including audience profiling using first-party data), in order to predict the likelihood a user will undertake the desired action (such as making a purchase or consuming multiple pages of content), which the advertiser will define. Then, using this target action as a predicate, the automation will submit a bid that scales in accordance with the likelihood that the users who did the search will undertake the target action.
This automation develops a machine learning model that typically requires a 14-day window to train in the field (with real-world data) so as to provide sufficient data to reach a high level of confidence in the changes that it is suggesting.
Now, this is a tried and tested technology that has been working well for millions of advertisers who use Google Ads for quite some years.
Automation in the Time of COVID
Recently, however, some advertisers have found this automation to be performing radically different from prior to the COVID-19 pandemic, and for some, this change has been really positive; for others, it has put them in quite a pickle and will take a fair amount of work to come back from.
Now, it’s not that Smart Bidding is broken or an update has made it worse all of a sudden, but rather that the machine learning component of this technology is unable to build confident models due to the highly diverse and rapidly changing inputs it’s receiving.
Limited Data Use
And this challenge is not scoped to just Smart Bidding or even Google for that matter. Facebook uses a similar model to optimise everything from ad serving through to budget management. Indeed, the Limited Data Use update in response to the CCPA has caused some undesirable performance changes, with cost per acquisition skyrocketing by 500% for Facebook advertisers.
Automation Adaptation
Now, just to be clear, I am not suggesting that automation that relies on Machine Learning and Artificial Intelligence is bad; in fact, I think that if we were to not have access to this kind of automation right now (during COVID), many advertisers would struggle to keep up with the rapid market and environmental changes.
Rather, the issue (as I see it) is that most advertisers are either not using this automation in a particularly meaningful way, or are not adapting their approach to ensure success in both the short and long term. Even if you are finding things are going well for you right now, my question is, how well equipped are you for the next radical change? And more so, are you willing to put your eggs in technology that is becoming more ‘black-boxed’ by the day?
And, believe it or not, this is where my talk finally begins!
Bridging the Gap
If you are using an ad technology that relies on anonymised data to train a machine learning model (like those in Google Ads, Facebook, Pinterest, Twitter to name just a few) you may need to seriously evaluate the sustainability of your approach.
So what can we do?
Don’t Panic
There is a lot of change right now, and that’s not to mention that which is still to come. And being prepared for this change is mostly about being ready to respond, even if you have not got a gameplan ready for that specific situation. Simon Sinek calls this “existential flexibility”.
“The capacity to make profound strategic shifts – 180-degree shifts – in order to better advance your organisation’s cause”.
Examples
Lindt is a great example of this — rather than closing their doors (56 physical stores) and waiting to see what happens, they took action and set up an e-commerce store, selling a range of their most popular products online or through kerbside pickup.
Primark, on the other hand, did not… and quite to their detriment.
All this to say, be ready for change — it is always happening even if it sometimes it feels like its not — from looming ‘death of the cookie’, through to the changing role of marketing automaton during and post-COVID.
Understand Your Data Landscape
So much can be gained from properly understanding what data your organisation has at its disposal, and how this data flows through the different touchpoints a customer and your organisation has with each other.
First-Party Data
This could be anything from:
- Customer records from your CRM or accounting systems
- Sales Pipeline data from platforms like Salesforce, Hubspot, or SugarCRM
- Customer survey data
- And Privileged Access Data, such as that which is associated with a user’s account when they are logged in to your website or app
All accessible from your first-party data pool. And, of course …
Third-Party Data
Aggregate data from:
- Your socials, including LinkedIn, Pinterest, and Twitter
- Web analytics data from platforms like Google Analytics and Yandex Metrica
- Marketing tech data from places like Facebook, Google Ads, and Klaviyo; and stats form web-based tech like Web Chat
- Public datasets like that from the Australian Bureau of Statistics, IBISWorld, or Kaggle
There is a plethora of data available beyond this, and we haven’t even mentioned 2nd party data! When you do map your data-sources, I highly encourage you to collaborate with as many teams and departments outside that of your own.
Siloed Data
It always blows me away how infrequently teams share data between departments — it almost never happens!
About 2 years ago, I was working with an international financial institution that was in the process of a data warehousing project and I was brought on as a consultant to guide the online process. During this project, we discovered that pretty much only the Sales Team, the Customer Service team, and the Accounts team knew that Salesforce was being used by the company — the marketing team didn’t know themselves when they found out.
That’s obviously quite an extreme example — but there is almost always data being collected that would be super useful to democratise within an organisation that nobody knows about or thinks to use.
+80% of Digital Data Created is Unstructured
And this data isn’t always structured in a way that makes it immediately useful — in fact, by most estimates, somewhere in the region of 80-90% of digital data is unstructured and can live in:
- Long-form text (think transcriptions, emails, customer contact notes, reviews and so on)
- Audio files (from a call centre or from a meeting)
- Pictures and images (and that can be anything from images of documents and barcodes, pictures of your customers or their products)
There are a lot of services that allow you to extract this information, such as Azure Media Services by Microsoft, which allows you to upload a video where it will build a text transcription from the audio (using voice to text), analyse the content of this transcription (signals like emotions, topics, brand terms and so on), as well as analysing the video stream (looking for changes in scene, faces that feature in the video content and so on).
This is a very powerful piece of technology that has proven invaluable, especially during COVID. I initially started using this to streamline writing documentation after workshops as it allowed me to jump to a specific speaker, search the transcription, and a timeline of topics that were discussed.
But you might not have a meaningful use case for all of these features, and rather access a single feature to solve a specific problem like Sentiment Analysis to better understand how your customers feel about your business in their emails to your customer service teams, for instance, or image processing to moderate content, or even to undertake some speculative demographic classification.
Data is Everywhere
Now, I appreciate things got nerdy pretty quickly so I’ll reel myself in before we go too far down the Machine Learning and Data rabbit hole.
And while it may sound all very futuristic, I bring this up to make an essential point: Data is everywhere — it’s just a matter of thinking creatively about how to access it and put it to work for your organisation.
And so, if there is anything to take away from this point, it’s to keep an open mind when you are looking at the data at your disposal. Start the conversation in your organisation now so that you can mitigate the impact (and — ideally — flourish) as the privacy measures we spoke about earlier begin to take a more serious hold.
Make Your Data Work For You
Once you have a good read on your data, it’s time to put it to work for you, and I don’t mean just for your reporting. So often, organisations put a load of resources and sometimes serious money into data collection for the purposes of reporting. The intelligence gained by these reports is then converted into actions which then need to be coordinated and dispatched.
But we now know that the rapidly changing environment we are living in right now will not wait. Think about this from a marketing perspective: by the time you have developed your findings and are ready to make changes in your marketing platforms, your intelligence is out of date.
Automation can help us here, especially that which is driven by Machine Learning and Artificial Intelligence — but, as we discussed, it can be subject to unpredictable changes in performance, especially when it depends on data points that are out of our control.
And so what it is really about is improving the quality of the data being used as an input for the automation itself. A staggering number of advertisers use their marketing tech completely out of the box, without enriching the data (or even knowing if it’s accurate for that matter) before using as an input in their automation suite.
And, as a result, they either have a mediocre performance from their marketing activity, or they have some early WINs but hit a glass ceiling which, no matter how hard they try, they can’t seem to push past.
Bridging the gap between your first-party data (CRM data) the anonymised data-pool that marketing tech vendors have access to (the third-party data), allows you to furnish this automation suite with data that is exclusively yours and not ‘black-boxed’ (and subject to changes outside your control).
All good in theory, but how might this be achieved in the real world? It’s actually far more achievable than most people realise.
Automation
I’m going to talk in hypothetical terms here, but it’s highly probable that some (if not most) applies to you.
This recipe requires three ingredients.
- A data source (in this case, a website)
- A transport mechanism (we’re using Google Tag Manager here)
- And some kind of marketing tech with ML automation capabilities (good luck finding a platform that doesn’t), in this case, Google Ads
Your Data Source
Let’s say a user goes to your site and, at some point, logs in — perhaps to complete a purchase or to manage their account. In the account creation process, you ask a user to provide a little bit of information about themselves like their age, gender, and (in this example) their favourite way to eat potatoes.
You want to be able to create specific ads for people who like their potatoes mashed vs those who like theirs baked. To achieve this, you need to create a ‘masher’ audience and a ‘baker’ audience. All well and good.
Your Transport Mechanism
And he — let’s call him Games Pickleson — writes a little bit of code that gets the website to tell Google Tag Manager how a logged-in user likes to eat their potatoes.
He uses Google Tag Manager to transport this data to Google Ads.
Your Marketing Tech With ML
And from there you can jump into Google Ads and create your audiences and apply them to your campaigns.
Now we used Google Ads in this scenario but you could swap this out for Facebook or Google Analytics — really, anywhere you want (provided the platforms Terms of Service allow you to).
And you don’t even need to have a website where you can log in. Instead, you might use your segmented email databases to send ‘mashed potato lovers’ and baked potato lovers’ emails, and you UTM tag the emails based on these audiences.
Then, when a user clicks on a link in your email and lands on the site, we can use Google Tag Manager to grab the audience out of your tag and transport it wherever you need it.
There are many ways to approach this — it just comes down to your creativity.
First-Party Automation
Just as a little bonus tip here, I highly encourage you to look into first-party automation. And Google Ads has a really awesome suite to develop these in.
Within the first month, CPC dropped 8.18% account-wide, allowing the account to get an additional 924 clicks which would have cost $2,032.80 based on the previous months CPC.
In the first 27 days of running the script in the account, Revenue Increased by 29.43%, while CPC dropped resulting in a ROAS Up-lift of 27.82%
The Great Divide: Insight to Action
- Don’t panic
- Understand your data
- Put your data to work
So there you have it. Three ways to cross the great divide and bridge the gap between first-party and third-party data.
