Data Blackbean Creative

The Science of Decision Making

Share on facebook
Share on linkedin
Share on twitter
Share on google

A couple weeks ago, we talked about the importance of knowing what the correct thing to do is, at least as it pertains to marketing. In that post, I touched on the value of data analysis when making decisions. This week, I wanted to do a quick case study of what data-driven decision making might look like, as opposed to purely insight (or guesswork) based decision making.

This example is based on a real experiment I ran with a client’s analytics and advertising data. For the sake of privacy, I’m going to obscure the details a bit. Let’s look at the steps I took to move from personal insights to data-informed decision-making.

The Hypothesis

Before I started work on any data analysis, I needed to determine two things: a goal and a hypothesis. The goal in this case was to improve lead quality and quantity delivered by a client’s Google Ads campaign, without substantially increasing costs. That’s a fairly broad and generic goal, but at least it’s measurable. If form submits and calls are more relevant and more frequent over a significant period of time, the goal has been reached.

The next part is really the most important, as it guides the whole analysis process: forming a hypothesis. A hypothesis, as you may remember from high school science class, is an explanation or proposed solution based on limited evidence — an educated guess. It’s the core component of the scientific method. We use hypotheses to frame and direct our inquiries, either proving the guess right or wrong depending on what the data reveals.

In this case, my hypothesis was as follows:

My client offers services on something of a subscription basis. I believe, for a variety of reasons, that their customers will be more willing and able to purchase services at the end of each month, after their second paycheque has been deposited but before their rent or mortgage payment is due. Should this be the case, adjusting ad bids during the final week of each month could improve performance.

This is a hypothesis with two parts to it. First, it asks whether my client’s customers convert better during certain times in each month. Second, it posits that increasing bids in the final week of each month will lead to improved campaign performance. In order to address the second component, we need to answer the first.

Interpreting the Data

Here’s where things get interesting.

Google Ads and Analytics provide so much data, available in many useful views. You can easily sort whatever data you’d like by day, week, month, or even year. What I wanted to explore was a little different. I needed to know, over the course of the past year, how much traffic was arriving on my client’s website through Google Ads, segmented by weeks in a month — meaning I didn’t care if it was week 1 or week 52 out of the year. I wanted to see how the first week of each month through the year compared to the second week in each month, and the third and fourth.

The data I wanted to view would be arranged like this:

Blackbean Chart

If I could pull data from the first whole month I have tracked through Analytics up to the latest full month, then sort it by weeks in months, I would be able to see how well each week in a month performed. If my hypothesis was correct, I’d be looking for big fourth weeks and small first and second weeks. If I was wrong, I might see the opposite, or see no distinctive changes at all.

But how do I go about segmenting my data by something like weeks in months? What that basically entails is dividing each month’s days into four equal parts, then assigning the first part to the label ‘First Week’, the next part to ‘Second Week’, and so on. Then, I’d need to count the views and conversions for each week and compare them. The problem is, I have 152 days of data to sort this way. Frankly, doing this manually would be a pain in the ass.

Instead, we turn to my new best friend, Python. Python is an easy-to-use programming language that’s become very popular for all sorts of projects, especially data analysis. Here’s where I’ll provide a disclaimer about my expertise: I’m new to Python. I played around with it a couple years ago, but I only returned to it seriously a few weeks ago. Take what I say about it with a grain of salt!

Boring Stuff for Nerds

This is what Python looks like:

Wow, exciting.

What this code does is open a spreadsheet downloaded from Google Analytics (I told you this would be fun!), then clean up the headings so it’s easy to compare the numbers. Once it’s all usable numbers. Once that’s done, it compares the number of users each day to the day of the month (ie, how many users did the client receive on the first day of each month? The second?), before finally grouping the days into weeks and printing the results.

An expanded version of this script included the conversions in the final results. Speaking of results…

I Was Wrong.

My hypothesis was exactly the opposite of the results. Like, it wasn’t a little off. I was barking up the wrong tree. And that’s really cool!

No, seriously. Traffic and conversions from paid channels were as much as 45% higher in the first week of each month than in the last. Each week after week one, traffic and conversions dropped in almost linear fashion. It turns out that, while I was right about weeks in months being a meaningful way to segment the data, my intuition was way off.

Why am I happy about it? Because it proves my point about data versus insights. Data doesn’t care if you believe something. It shows you the unbiased truth. If I’d taken a quick skim through the data, made an educated guess, and acted on it, I’d have done the opposite of what I ought to. Since I took the time to develop and test a hypothesis, I can proceed knowing that I’m on the right track.

Well, kind of.

It’s More Complicated Than That

My hypothesis was based on the cyclical nature of finance in the average worker’s life. For many middle and lower income families, purchases are made based on paycheques. Your overall wealth might be fairly steady, but the fact is that for many, you’re cash-rich right after a paycheque and cash-poor just before the next. I believed that was a motivating factor for my client’s customers.

The fact that their behaviour is the opposite of my hypothesis means that I was wrong, not just about when they’re eager to spend, but also about why. To a certain extent, why is less important to know than whether — a shop owner might not care why they’re so busy on Mondays, but would certainly care whether that trend was repeatable and actionable.

On the other hand, knowing why consumers act the way they do is important if you want to maximize the effectiveness of your marketing. If consumers are purchasing far more in week one than week four, there’s a reason. That reason could be a lot of things. Is my budget low by the end of the month, so paid conversions drop? Are consumers motivated to purchase after their necessities (rent, utilities) are taken care of, which for many is at the start of the month?

To answer those questions, I have more research and analysis to do. What’s exciting about digging this deep into an issue is the completeness with which you can understand a business. For many small businesses, their marketing is guesswork. Success might be chalked up to hard work and a positive attitude. Failure might be blamed on bad luck, or ‘the economy’. In many cases, the real answers are there, hiding in plain sight. You just might need to learn Python to find them.

Leave a Comment

Your email address will not be published. Required fields are marked *