The Strategists’ Corner: Building modern measurement stacks

Announcer: Welcome to the Ignite Podcast, the only healthcare marketing podcast that digs into the digital strategies and tactics that help you accelerate growth. Each week, Cardinals experts explore innovative ways to build your digital presence and attract more patients. Buckle up for another episode of Ignite.

Rich Briddock: Hello, everybody, and welcome to another episode of the Ignite Podcast, another segment of the Strategist Corner with Ben Dutter. If you’ve heard our first episode, we high-level covered a number of topics about what healthcare marketing could learn from other verticals that Power and Ben have a ton of experience in. Today, we’re going to go a little bit deeper into testing, incrementality, and really around how you take those new concepts as a marketer team into the organization where potentially there’s a lot of ingrained ideas of how things are done and what channels deliver, and you start to shift that perception and shift the approach.

Then, obviously, this is, again, something that you’ve dealt with a lot over many years, probably more than you would have wanted to as the agency partner. What we hear from clients all the time is, no disservice to any of our clients, but oftentimes you hear two contradictory statements, which is, “It’s not working, but I know it works. It’s not working, but I know it works.”

Then we have to come in and say, “Actually, it might not be working because of this, and we need to change the approach.” That is something far more radical than they’ve seen over maybe the 10, 15 years that they’ve been doing digital marketing. First of all, talk to us a little bit about what these changes are that you’re coming in and recommending in terms of approach, and then we can get into how you sell that in to the exact level and the boardroom.

Ben Dutter: I joke that 95% of my job is just group therapy for my clients. A lot of it is just change management. Really what we’re talking about is risk versus reward. What’s the organization’s risk tolerance level? A lot of organizations have a very low risk tolerance. They are, I would argue, overly fearful. They’re afraid that if something goes wrong or they stop doing this thing or they start doing this other new thing, things are going to spiral out of control. It’s safer for them to just try to tweak and optimize within this small box that they’ve constructed over the years.

How do you do that? First, I think a lot of it has to do with provocation. A lot of people will say they want to start with education and make it, “Hey, did you know that you can do this?” That’s certainly a path, but if I’m getting brought in, it’s probably we’re well past the point of no return, where the business is already struggling from a performance perspective. They need someone to be a little more critical and tell the harsh truth to the boardroom and the CFO.

The provocation in many cases is what percentage of all of your X revenue, customers, leads, whatever it is, do you think come from paid? Let’s just take a survey around the boardroom, around the C-suite. Some say 10%, some say 0%, some say 100%, and everything in between. It’s like the fact that everybody has a different answer tells me that we don’t know. Even worse, they might say, “Oh, we know for a fact it’s a set percentage.”

If everybody agrees, everybody says 50%, that’s just as much of a red flag as everybody being on different pages, because it tells me that they’re probably looking at one measurement type, and everybody is just trusting that’s accurate, when in reality, the contribution versus the baseline changes every day, week, month, quarter, year. The right answer should be, “Depends on what timeline you’re looking at, what time period.” I’m talking about this business unit versus this business unit. The answer is always more nuanced.

Step one, to me, is provocation. Getting that group therapy, confrontation, conversation started. Let everybody put out the truth of what they see on the table and illustrate that there is probably not alignment. Once you’ve started that, then it opens people up. They become more curious around, “How do we determine the actual percentage?” That is when they’re willing to learn and take some risks, because you’ve already undermined the confidence that they have in the status quo, and that’s really when they’re more risk-tolerant, and you can introduce some of these new concepts that we’ll get into.

Rich: Once you peak the curiosity, then what are the key concepts that we’re introducing, and in which order are we introducing them in?

Ben: We follow a framework of highest authority to lowest authority in terms of what are the measures that we care about as an organization. We like an acronym at Power, which is easier to remember. We follow the BEATS framework, B-E-A-T-S. I’ll go through each one as quickly as I can. Don’t get bored, please.

B is your business metrics. We talk about the single source of truth is the P&L. If you’re not driving contribution margin, if you’re not improving your overall revenue and you’re spending more, something ain’t adding up. I don’t need any more fancy measurement methodology to tell me my media is not driving incrementality if I’ve doubled my spend and my leads are flat. That’s good enough. I know that something is broken. There’s something wrong. That’s number one, most highest authority.

Number two is experiments, which we talk a little bit about in another episode. How do you do these test and control experiments or AB experiments where you’re showing ads to some group of people and not ads to some other group of people? Does that prove that there’s actual incrementality or lift in the exposed group, whether that’s a revenue, a lead, a brand lift association, whatever it is, because of the statistical rig, the next highest authority behind a business P&L?

A is your typical modeling analysis. Think of marketing mix models. Think of just looking at trend lines, even regression analysis, whatever flavor you want to pick. You’re looking at, we did some stuff. Did some better stuff happen downstream in the aggregate? Then the last two are the two that most brands overemphasize, I think, and so this flips it on its head. T stands for technology. A lot of brands focus on the tech attribution. They’re looking at an MTA, a multi-tech attribution tool. They’re looking at Google Analytics. They’re looking at a CDP. They’re looking at a CRM. They’re trying to tie every single deterministic touch point.

Ben was exposed to these five ads, landed on this page, clicked this button, and made this conversion. They put way too much emphasis on that. The technology, one, is regulated. I’m talking to a healthcare audience right now. That is going to only continue to get worse probably. Two, it’s flawed in many cases by stuff that you can’t track. There’s no way to technologically track, “Hey, I got this referral from my buddy who had the same issue.”

Rich: Even wool gardens on Facebook and TikTok, and how do you title together unless you’ve got a tech budget?

Ben: Exactly. That’s why it’s so way on the list. The final one is customer surveys, which I actually think are helpful directionally, but customers have a very flawed recollection. It’s always human and psychological. It’s an interesting data point, but I wouldn’t build my whole marketing strategy around it. That order between starting at the most authoritative, the business, moving through data science techniques and experiments, and then finally, what you would consider traditional performance marketing metrics, that order is super important. Coming up with that full system or ecosystem of measures and presenting that to a board can showcase the “so what” very easily.

Rich: That’s reframing the approach and de-emphasizing things that have probably been over-emphasized and emphasizing things that probably have not even been thought about because they’re probably thinking about B, but they’re almost certainly not thinking about E and elements of A.

Ben: That’s right.

Rich: How do you then go in and prove that out? You present BEATS, and I am a C-level executive at a PE-back provider group, and I’m saying, “We’re not getting what we need to get from digital. I don’t know what we are getting, as we’ve already covered, but I know we’re not getting enough because my B doesn’t show it. My business documentation, my P&L, is not where it needs to be.” How do you then go in with those elements and start to show improvement? Where would you start? What would the recommendation be?

Ben: Thankfully, within the A framework here of analysis, there is a marketing mix model. A marketing mix model or an MMM is a backwards-looking statistical analysis that looks at all of the inputs associated with marketing. Easiest way to think about that is spend, daily media spend by platform, for example. You can run that through an MMM. There’s open-source MMM out there, but there’s also providers that handle it for you.

They will come back with hypotheses. That MMM will say, “Your meta was this effective, your Google was this effective, your TikTok was this effective.” That’s probably very different. Those numbers are very different than what the org sources as truth today. It might flip it on its head. It might say something like, “Search is the least effective channel,” when they internally think search is the most effective channel, is a common occurrence.

Bringing that hypothesis engine to them, that analysis, and saying, “Hey, what if this is true?” that, again, it’s provocative. It peaks that curiosity. If I’m wrong here, I could be wasting millions or tens of millions of dollars depending on the size of the business. Then they’re like, “So what? How do we prove this out?” This is just fancy math. What do you do with it? That’s where the E comes in, the experiments.

We say, “You know what? We don’t want to hurt your quota on your volume. We don’t want to hurt your performance. However, we’ve identified these handful of geographies, maybe a DMA or a Metro, or what have you. We’re going to turn media off, and we’re just going to observe just those dark tactics for a while.

If we don’t see a precipitous drop, then we know that we’re overinvested in our media. That always piques the interest of the CFO and the CEO because they’re usually held to an even target of some kind, especially if they’re PE-backed, in your earlier example. The one-two punch is show them the what-if with a credible, data science-backed model, and then prove the model right or wrong. Either way, you’ve learned something.

Rich: Then you get in front of the CFO. We’ve got this MMM output. It’s suggesting that maybe we’re over-investing in certain channels. We run a holdout test because that’s the easiest one to get buy-in. Spend less, and we’ll see what happens. They do spend less, and there isn’t this precipitous drop. How do they then take the E and the experimentation and then fold that into the marketing strategy moving forward so that they’re not just learning that, “Oh, search isn’t actually as great as I thought it was?” How are they then learning where they should invest, and how are they constantly evolving their marketing efforts through experimentation to drive a better ROI ultimately?

Ben: It comes back. It’s really the same process, just in reverse. That MMM analysis often will show a channel or a tactic that is more performant than they think it is. Maybe it’s TikTok in this analogy. I’m talking about TikTok today. With TikTok, maybe they think internally, the client thinks that it’s a $1 return, and the model actually suggests that it’s a $4 return. That’s four times more incremental than they think it is.

The easiest way to prove that is also with an experiment. We’re going to heavy up TikTok in only certain geographies, the same way that we did a holdout test in certain geographies. We’re going to 5X, 10X the budget of TikTok in certain geographies and see, does that 4X return hold true at that increased spend level? I’m being hyperbolic with my percentages, but you see what my point. That’s one way. That same process, just in reverse.

The second is, ideally, you have some kind of understanding of who your customer is, and you know where your customer spends time online, whether it’s traditional or digital media, same thing. Where are they influenced the most from a media mix perspective? Of course, there’s the obvious choices like Meta and TikTok and Google like we keep talking about, but there’s ones that are less obvious. Maybe they spend a lot of time listening to podcasts. Maybe they spend a lot of time on Reddit or Snapchat or whatever.

Finding that research that gives you another data point that says, “Hey, our audience looks like this, and they spend a lot of their time on Reddit. We have a hypothesis that if we introduce ads on Reddit, we will see improved performance.” That’s another example of an experiment where you can have a methodical test structure like that and a testing roadmap for the next probably 12 to 24 months that you’re trying to answer a series of questions.

Rich: How do you deal with the objection, which I’ve heard in the past, and I’m sure you have as well, which is the model is showing that TikTok is performing better than search or Meta is performing better than search? That just reinforces the bias that I already had or the opinion that I already had that you’re not running my search very well. That’s the reason why search isn’t performing strongly. Actually, I still believe in search, but I doubt the way that you’re managing my search campaign.

Ben: It’s a common challenge, I think. What makes a tactic incremental is complicated, and it’s also a snapshot in time. Like we said earlier, your performance will change over time. You can do everything else the same, but the market moves around you. Your competitors change, their seasonality. There’s forces in the world that influence consumer behavior or patient behavior in your case. What that means is that it very well could be that what we were running six months ago was the right tactical deployment, but it’s not right now.

Usually, there’s some other corroborating data points that you can point to that if we’ve made or not made, in some cases, tactical shifts that correlate with this decline in performance, then you can start to tie back that time zone. For example, T is still important. That’s where the tech tracking still comes into play. Has our attributed cost per lead gotten worse over this period of time that the model is looking at? If that data is also showing that it’s gotten worse, and our MMM is showing that it’s gotten worse, it very well could be that we’re running it poorly. It could be that the media is poor.

Now, let’s say that’s not true. Say that all signals just show that the channel is not a good fit, then it ends up becoming, how do we prove that? Usually, you’d want to bring in some kind of third-party subject matter expert, consultant to contractor, another agency, someone internally at the brand, and they can run an audit or they can run a cook-off. They’re going to run their own version of the campaign, and it comes back to that experimentation. I think a good agency partner is very open and transparent with that.

We talk a lot about at Power, we’re a glass box. Whatever you guys want to look at, here you go. We hide nothing, and we welcome the challenge of another agency or partner coming in and looking at how we’re set up and how we’re executing. I think even if we were wrong, let’s say in that example, someone came in and said, “Hey, you guys are doing it wrong,” even if we’re wrong, my ethical imperative is to my client. I want to know that. I would want to learn something now. Then we take that learning and we incorporate that into our new SLP for that client or broader.

Rich: I think that’s exactly right. What we’re talking about here, and I know we’re talking about it at a very high level, can be incredibly complicated. We’re talking about Mosai, we’re talking about modeling, we’re talking about different incrementality testing, market matching, all these various things, holdouts, et cetera.

You mentioned provocation, but once you’ve provoked the desire of the boardroom or the C-level stakeholders to be invested in this new approach, how do you help to educate them as a marketing leader in an organization? How do you go about educating them so that they understand and can feel comfortable with what’s happening, or do they not even need to feel comfortable with what’s happening, as long as the outcome is the desired outcome?

Ben: I think it’s a good question. I think it depends, again, based on the organization’s culture and their risk tolerance. If you have a very high-risk tolerance, very high-trust C-suite, for example, they probably don’t need to know. They just need to see the P&L. P&L is getting better. Great. We cut ad spend 20% and maintain lead volume. Awesome. That’s great. Then what are we going to do with that extra bottom line ad that we generated? That’s not the case in probably 95% of organizations that I’ve worked with, where they’re extremely risk tolerant and extremely high trust. It’s usually the inverse. They’re very risk-averse, and they’re very micromanagy, unfortunately.

What ends up happening is I have to be extremely clear with expectation setting for all parties involved. This is why I call it group therapy. I have to go back and talk to each stakeholder one-on-one, not in a big 10-person Zoom call, one-on-one, saying, “What do you want to get out of this, Mr. CEO? What do you want to get out of this, Mr. CFO?” and go through that process with them. They say, “Oh, I just want to know if my blah, blidy, blah works.” Great. I’m going to write that down.

Once I’ve collected all that, I make a very tight and clear testing brief. These are the three questions that we’re trying to answer. It’s either a yes or no. That’s it. We make that extremely clear. Here’s how long it’s going to take. Here’s the levers we’re going to pull. Here’s what we do as a contingency in case stuff goes sideways in the test, and make that really, really clear and memorialize it as a document that everybody has signed off on. Google comments were great. They say “approve.” We have a little timestamp for them. That then locks everybody in. We’re all arm in arm. We’ve committed down this train, and we know exactly what to expect.

Rich: Speaking of briefs, is there such thing as a testing roadmap that helps to prioritize this experimentation and create the shared understanding with the client and the agency that’s running these experiments?

Ben: I think that’s critical. That’s really what we’re talking about here, is getting this centralized document that everybody has visibility into, everybody has bought into. There might be links out to more technical explanations or whatever, but there’s an exact summary of some kind. A CEO can pull it up and say, “What are we testing right now?” They see a little green–

Rich: And why?

Ben: And why? They see this is active, and they got it, and they’re clear on what the timeline is. Nothing erodes trust more than you not doing what you said you were going to do or things going differently than you said they were going to go. You have to stick to the plan, and you have to set the expectations for all parties, whether you’re an internal marketing stakeholder to a C-suite or you’re an agency partner. I think that’s 101.

Rich: A lot of the things we’re talking about, certainly the modeling stuff, is great for the CFO. It’s great for the CEO. It works at that centralized level where there’s a ton of data. In healthcare, oftentimes, we might be dealing with a provider that’s at a single location. We might be dealing with a house of brands where a provider has been rolled into an MSO or a DSO, and their concern is their location or their group of locations.

MMM might be tricky for that because there might not be enough transaction volume to really run MMM and get the kind of statistical significance that we’d need to have any kind of confidence in the data that we’re seeing. What can we do to change the paradigm, the approach, at that level, at that more granular level for those folks if the status quo isn’t working and delivering the results that they need?

Ben: There’s two angles to that question because one is execution. Is there something that we’re doing that we’re not executing well? Is the media or the creative not performant? There are plenty of leading indicators, I would say, old school performance marketing indicators in that case that can tell you maybe not the truth but the direction towards the truth. Something as simple as, “Am I seeing a high degree of customers respond in a ‘how did you hear about a survey?’ say, ‘Oh, yes, I saw your ad on TikTok’?” That alone is directionally helpful, regardless of however big your business is.

To make it a little bit more scientific, though, there’s a concept of ladder flighting, which is just a jargony word that just means we’re going to hold media in a certain budget or mix for a certain flight of time regardless of whatever the attributed performance is telling us. We’re just going to keep it as even peanut butter spread for a month or for a quarter or whatever makes sense for the business that they’re in. Then that’s going to establish us on a baseline. We’re going to see, every day, the cost per lead goes up and down a little by little, but we see it plateau after a while.

Now we’re going to introduce some kind of change, either a step change up, we’re moving up the ladder one rung, or we’re going down the ladder one rung. We’re going to introduce a 10% increase on total media budget, or we’re going to cut by 10%, or we’re going to introduce a net new channel, or we’re going to cut a channel, whatever it is, and we’re going to do that same process again. It’s just like, imagine if you’re trying to exercise. We’re talking about health here, right?

Rich: Yes.

Ben: So to exercise more. If you’re trying to exercise, how do you learn if something is working or not? You have no test and control, really, on your own body unless you’re doing a right arm, left arm. It’s really just before and after. You want to try to hold as many variables steady as you can. If I’m never going to the gym, and then I suddenly start going to the gym five days a week, and I look a lot better and I feel a lot better, pretty good indication that the gym is doing something for me, right?

Rich: Yes.

Ben: You just have to make a meaningful enough step change to be able to feel confident that it’s affecting the overall business.

Rich: That’s the key piece, meaningful enough. Minimum detectable effect, we’d say in the CRO world. If you’re not Amazon, don’t just try and change the call now button from green to orange because you won’t notice any difference. It has to be a meaningful change in order for you to actually see. I think that’s what we often see, is people are like, “I threw another $500 at it, and it didn’t work,” and maybe their CPA is $500. The chance of them even getting one conversion wasn’t even a given at that rate. I think that’s the key piece, is it does have to be incremental enough to be detectable.

Ben: Totally. There’s some statistics calculations, power analysis, for example, that will help you figure out what that MDE is. I would say this is a good rule of thumb. At least 10% change of whatever the input variable is usually a good starting point. If it’s your total medium mix, if it’s your total conversion volume, whatever it is, 10% change starts to become quite noticeable on a baseline over a long enough period of time. If it’s 30 days, maybe not, but 90 days, certainly. That requires some courage and some intestinal fortitude, so to speak, on being willing to suffer for 90 days to learn that insight. It’s important because that will inform the next maybe 10 years of how you execute on your media.

Rich: All of this is about willing to embrace change, having that intestinal fortitude, as you put it. That’s also healthcare-related, so we can talk about that.

Ben: There you go.

Rich: All right. Ben, this has been fascinating. You heard it from Ben. Be more provocative in the boardroom.

Ben: That’s right.

Rich: That’s the way to win in terms of change management. Thanks for sharing these nuggets of wisdom with us. We hope you will join us for another episode of Strategist Corner coming to you soon.

Ben: Thanks, Rich.

Rich: Thanks, guys.

Announcer: Thanks for listening to this episode of Ignite. Interested in keeping up with the latest trends in healthcare marketing? Subscribe to our podcast and leave a rating and review. For more healthcare marketing tips, visit our blog at cardinaldigitalmarketing.com.