Aaron Zagha
The key to any success in CRO is making sure that you design tests in a way that you can't hit significance, or just make the change and not care about it.
Chase Clymer
Welcome to Honest Ecommerce, a podcast dedicated to cutting through the BS and finding actionable advice for online store owners. I'm your host, Chase Clymer. And I believe running a direct-to-consumer brand does not have to be complicated or a guessing game.
On this podcast, we interview founders and experts who are putting in the work and creating real results.
I also share my own insights from running our top Shopify consultancy, Electric Eye. We cut the fluff in favor of facts to help you grow your Ecommerce business.
Let's get on with the show.
Chase Clymer
Hey everybody, welcome back to another episode of Honest Ecommerce. Today, I'm welcoming to show the CMO at Newton Baby, Aaron Zagha. How are you doing?
Aaron Zagha
Good. How are you?
Chase Clymer
I'm doing great. So Newton Baby, the largest direct consumer company in the baby sleep category, what is the day-to-day life for you as the chief marketing officer?
Aaron Zagha
Yeah. So I oversee everything marketing-related, plus customer service. And I also work pretty closely on new product development. Yeah, we're a good-sized team, mostly Z2C, a little wholesale.
Chase Clymer
Awesome. Well, take me back in time. How did you switch career paths from investment banking to ending up in the Ecommerce space?
Aaron Zagha
Yeah. The long and the short of it is that I love LA. If I wanted to work in LA, I needed to figure out something other than investment banking. There's not much of it here. Plus, I didn't really like it all that much. So I ended up working in-house in an internal M&A Group at The Wonderful Company, which owns pistachios, all the pistachios in FIJI Water and Teleflora. And typically, when the operating companies needed people really quick to fill in because the exec left.They barred people from the M&A Group. And so something came up with Teleflora to fill in the head of Ecommerce there left kind of suddenly. And I was the only one in the group that had previous tech startup experience. So they threw me into Teleflora to run things on an interim basis while they were recruited.
And it worked out pretty well. They ended up hiring a CMO for Teleflora US. But they offered for me to run Teleflora's international Ecommerce and realised that there wasn't much of a future in finance in LA. And then I did Ecommerce and tech. I pivoted career-wise towards that direction at that point.
Chase Clymer
That's amazing. And so along the way, before you ended up at Newton Baby, any other notable brands?
Aaron Zagha
Yes, so Teleflora, selling flowers online, mostly in Australia, New Zealand, and the UK was the main one. But prior to that, I had worked on a tech startup called, well, first it was called Shop Squad and we pivoted to own that. This was back in 2010, 11, 12. It was a kind of social commerce, peer-to-peer live shopping platform that was founded by a bunch of big tech entrepreneurs.
Chase Clymer
That's amazing. And how did the Newton baby opportunity come to you?
Aaron Zagha
Yes, I've been the founder for years. He had actually made a job offer when I was debating between Shop Squad and his company was for a different company. was running in time. And they're both startups. And the most important thing in a startup is making sure you get along with people. So I offered each of them to work for free for a week to figure out the fit piece. I ended up getting offers of both.
I turned down my current boss and ended up taking the Shop Squad offer. But he always kept in touch and as both of our careers pivoted to Ecommerce, we stayed in touch. And he asked me to come aboard at Newton once they had gotten to a size that merited the CMO.
Chase Clymer
Absolutely. And what year was that?
Aaron Zagha
Oh that was like seven years ago. So 2018-19. I started advising first and then joined full-time as the in-house CMO a year later.
Chase Clymer
What were some of those roles that he was looking for a position to be filled? What were you coming on board to take off of his plate?
Aaron Zagha
Yes, they had a few of the common running things. Newton at that point was 3 or 4 years old. They're doing a couple million a year in sales, but really not scaling very quickly. And so at first, he was, I think, really looking for an outside opinion on how to scale.
And I looked at his marketing channel mix and had some critiques and recommended some changes. And so those changes and recommendations I made ended up working pretty well and business started taking off to the point that he could actually afford a full-time in-house CMO.
Chase Clymer
Absolutely. And now, did your background with finance and numbers help you with the marketing work that you helped build over there?
Aaron Zagha
Yeah. So my DNA is really a performance marketer. One of the things I love about Ecommerce these days is that everything's trackable and measurable.
Chase Clymer
They say.
Aaron Zagha
They say. Well, to some degree, you can get stats.
Chase Clymer
It's infinitely better. You can be a naysayer. There's data and it works.
Aaron Zagha
I think so. Certainly in the way that I run the business corroborates that, I think. But yeah, so my financial background and particular statistics really come in handy for Ecommerce. Understanding statistical significance is one of the most important concepts if you are testing, iterating, doing any kind of conversion rate optimization, really even just understanding ad performance. So I credit much of my success to having a financial background and specifically understanding statistics.
Chase Clymer
Absolutely. For the listeners out there, right? I bump into this and I'm going to try to ask this question in a way that isn't biased. Everybody wants to do conversion rate optimization, but hitting statistical significance is hard to do until a certain size.
I'm done with laying. It's very obvious what I think. But what would you say to younger brands when they want to really get into split testing and CRO?
Aaron Zagha
Yeah, it's such a great point. I couldn't agree more wholeheartedly. And I mean, we're of a size and we never hit statistical significance. Like, you know, I think there's so many different levers that either make it worthwhile or not worthwhile. Size is definitely one of them. The size of the swing you take is another one. And the way your business works. So it was really frustrating. I mean, we did very little CRO in the early days.
And even back then, we were probably of a size where it could have worked, but we do a lot of our selling off-site. So influencers on top of the funnel have always been huge for us. By the time someone comes to the site, they're pretty much decided. So we did full site redesigns, and completely different websites didn't hit significance. I mean, like crazy things. So it's not only, are you making a huge change that could actually change consumer behavior.
It's also by the time someone comes to your site, are they influenceable or have they already decided? So there's so many different pieces of that puzzle. But yeah, I mean, the key to any success in CRO is making sure that you design tests in a way that you can't hit significance or just make the change and not care about it.
You know, kind of like just see what happens because chances are you're not going to screw things up terribly if you put any thought into your site and, you know, kind of have experimented at all. But yeah, designing tests to make sure that they hit significance and using sample size calculators is really key if you want to take that statistical approach.
Chase Clymer
Yeah. I have these conversations all the time. Getting to $1 million a year as an Ecommerce brand is no easy feat. The first million is the hardest. But even around there, unless you have a super low average order value, if you're doing conversion rate optimization, optimizing for the conversion, aka the purchase.
That test is going to be running for weeks, if not months. Now your traffic has changed because Facebook's algorithm has changed, your marketing messages have changed, the holiday has changed. So sure, you can run a split test and it'll hit stat sick at some point. But all these other factors changed and now it doesn't. Now you could argue it's invalidated.
Aaron Zagha
100%. I mean, there's so many pieces to that. And so then do you go up the funnel to see how many data points you go to add to cart? Do you go to minimizing bounces? But do those even correlate with purchases? Maybe, maybe not.
Chase Clymer
I just say listen to smart people, go do user tests, see. Ask your customers what they don't like. You can mine your reviews, you can mine your customer support tickets. All that stuff is valid. Do you need to run a test? Not at your size.
Wait, I'd say 5 million years. You can start to get a test result in a week. But 9 of the tests are going to be meaningless. That one test though. That one's awesome.
Aaron Zagha
I totally agree. I mean, you can even run your tests in your ads maybe. Somewhere else where you'll get the significance faster. But I totally agree. If you're not a couple million years in revenue, there's no way.
Chase Clymer
Have they already made that decision to buy before they even hit the website? And if you have an amazing offer and product and positioning with your advertising, like one, you're probably just turning $1 into $5 left hand over fist.
But like, two, yeah, then your experience on site doesn't matter quite as much. I've seen dogshit sites sell $10 million a year because the product, the positioning, the offer, and who they're talking to in the marketplace just all resonated.
Aaron Zagha
That's right. Think about DRTV in the old days, right? There wasn't a website. People just like, so if your ad is good, people just call and order over the phone. There was no website.
Chase Clymer
Absolutely.
Aaron Zagha
It didn't even matter.
Chase Clymer
“I have been in business for nearly 20 years, and very few companies I have hired in that time have performed as well as Electric Eye. They have knowledgeable staff, and our project was delivered on time and on budget. Electric Eye has exceeded my expectations, and I look forward to working with them again.”
That is a direct quote from one of our clients at Electric Eye.
Electric Eye is a Shopify Plus partner that has helped over 100 brands migrate, redesign, and optimize their stores since 2016.
If you'd like to increase your conversion rate, average order value, and revenue per session, we are the true Shopify experts you've been looking for.
Right now, we're offering a free diagnostic to qualified brands that reach out and mention the podcast. Visit electriceye.io today to schedule a call and send us a message. Find out why we have over 50 5-star reviews in the Shopify Partner Directory. Again, that's electriceye.io. E-L-E-C-T-R-I-C-E-Y-E.io
Let's be real. Sales tax is a nightmare. You didn't start an Ecommerce brand to stay up late googling tax laws or stressing over whether you filed in the right state. If you're still trying to DIY your sales tax, or worse, using a tool that leaves most of the work on your plate, you need to check out Zamp.
Zamp is a fully managed sales tax solution built specifically for Ecommerce brands. That means actual experts handle everything. Registration, filings, remittances, all of it. Brands like Tushy, Glamnetic, and David are already using Zamp to get sales tax off their plate. Some Zamp customers spend as little as 2 minutes a month thinking about tax. Imagine that.
No confusing pricing tiers, no surprise fees, just one flat rate to make the entire mess disappear. And yes, it works seamlessly with Shopify, BigCommerce, QuickBooks, and more. So if you're tired of sales tax headaches and want to stop risking penalties or audits, go to zamp.com.slash.honest and get 3 months free. That's Z-A-M-P-dot com slash H-O-N-E-S-T https://zamp.com/honest.
Chase Clymer
Alright. So I love talking to CRO, but we're not here to talk about that. We're here to talk about Newton Baby and what you've done there. So one of the things that I know about your role is you help educate and hire new team members.
What's one of the first things that you discuss with your team members?
Aaron Zagha
I'm not going to go right back to it. But it is of statistical significance. There are two things. Number one, I make sure that everyone understands statistics. Chances are, if they're on the marketing team, they're going to need to iterate whether it's on ad creative or email subject lines, whatever it is. They're going to need to understand basically just how to judge winners, how to pick winners. And it's not always intuitive.
I want to make sure that they're not making decisions based on bad signs. That's one thing. But the second thing is attribution. And it's something that I talk about all day, all the time. But making sure that you know how to judge a winner and that you're using the right data to do that. And it's not the last click for us. If you have a considered purchase where there are multiple touch points, it's definitely not going to be the last click. It's not going to be even a multi-touch attribution for us. And it's not even medium and spotty either. For us, almost always the gold standard at this point is incrementality testing. And so basically, just making sure that my team understands all the different types of attribution, what to use when, what is reliable when, is the second thing that we talk about after statistics.
Chase Clymer
Yeah. And I think that just teaching them about statistics and about iterating that almost gives them the freedom to fail. It's like, all right. It wasn't wrong. You just learned which one won't work. And it helps them shift their mindset.
Aaron Zagha
There's a lot of companies and I've been thinking about this more and more that give out awards for the worst failures. But you win something for a big, and it's really just giving that permission to take the swings and take chances.
I think it's super important. I have a marketing scrum call every Monday and my whole team's on it with the CEO also. And 10 times now, I've brought up points or case studies or whatever about companies that made ugly ads and how successful they've been. The CEO has a meltdown every time I say, we're not making ugly ads. I don't actually mean it doesn't have to be ugly. It has to be different. You need to take the chance or you're never gonna have a breakout winner.
Chase Clymer
No, no, no.
Aaron Zagha
And giving them that permission to fail, yeah. And take the series.
Chase Clymer
It goes back to it. It isn't about the polish. I think that's part of Apple's brand. But it's beyond the polish of their brand. Their positioning, their copy. A thousand songs in your pocket was an insane tagline back in the day. And they could have done that with a shittier thing and it still would have sold.
Aaron Zagha
Yeah. And again, good luck trying to be Apple. Don't. Oh, my.
Chase Clymer
Yeah, that's my favorite thing. Obviously, I have an agency and we're always talking. All right, we're talking about redesigning a website or doing a landing page strategy. And we're like, all right, well, send us over some of your competitions. And we get Nike and I'm like, this isn't your competition.
Aaron Zagha
Not your competition.
Chase Clymer
These are household names. Grandparents with dementia remember these brands more than their children at times. What are you talking about?
Aaron Zagha
Exactly. Unless you're super luxury and the highest price point. Forget about Polish. It's not going to sell. We are luxury. It's kind of ironic. We are at that end of the spectrum for baby stuff. But we're selling baby stuff.
We sell things that get cooped on all day. So humanizing that luxury is a key to our success also. so thankfully, because of the category, we do have permission to go unpolished even though we're at the expensive range of pricing.
Chase Clymer
Absolutely. Maybe this is a question in two parts. There's a lot of new technology coming out these days making things “easier”, obviously, AI and all the stuff it can do for you. But still today. What do you think is one of the hardest things that brands have to tackle today in the marketing world?
Aaron Zagha
I mean I think creativity is still probably the hardest. Let's get specific. If you're active on TikTok, you have to be very creative. It's crazy. And creativity is a thing that isn't as measurable. By definition, creativity is like that. It's just a tech science measurement. There are tools that do it with AI and whatever else.
You still have to come up with it and have to generate it. And so if you're selling a toaster that sits on counters, maybe you can AI generate 7,000 different images. But we sell things that are babies and AI is notoriously the worst at generating babies.
Chase Clymer
Yeah.
Aaron Zagha
We're messing around with it and we use a ton of AI across all kinds of applications. But creativity is so hard and it really isn't there yet from the Gen AI perspective. It's getting there maybe next year. I don't know. Creative generation and testing is challenging no matter what. If you're scaled up, and especially on TikTok.
Chase Clymer
Now I'm going to ask you another question. So you said you're using it all over the place. What would you say is a unique area where you guys have been using AI that maybe a brand hasn't thought of yet?
Aaron Zagha
So I think one of the tools that we use is an AI-based CRO platform where we don't do A/B tests anymore. Ironic given the start of this conversation. But it's called Evolve AI and it's awesome. Essentially, you feed it a building blocks column, which maybe in another world would be A/B tests, but you give them a bunch of different options and they use every data point they have based on the customer.
So, you know, source, path, geo, whatever else to then assemble the formula of building blocks of the site to maximize conversion rate. So, you'll never see the same homepage twice, for example. Like if you haven't converted on it the first time, they're gonna show you something else the second time if they recognize you.
And what happens is they turn off losing variants as things definitively lose and hit significance. But it's not an A/B test, like it never concludes. It's always iterating based on however much data you can collect in real time.
So it's interesting in that like, it's not like, this cartridge went out over this cartridge. Both cartridges are still alive. It's just that maybe someone in Miami on mobile, this one works better. And on a desktop in LA, this one works better.
Chase Clymer
Yeah.
Aaron Zagha
So that's a really interesting machine learning type AI tool that we're using. It still hasn't found too many standout, slam dunk, statistical winners. But we are seeing some marginal gains there. And I haven't heard that much about using this yet.
Chase Clymer
How long have you been using that platform now?
Aaron Zagha
That's probably been about a year.
Chase Clymer
Oh yeah. Interesting. And you guys are still on Shopify on the backend though?
Aaron Zagha
Yeah.
Chase Clymer
Oh yeah. I'm sure there's going to be some people that look into that and explore it. But I do want to ask, it still doesn't sound like a set it and forget it solution. You're feeding it building blocks, as you said.
Aaron Zagha
Yeah. It also makes suggestions for things that you should test. So because it's basically observing all the behavior on your site, it'll notice friction points and say, oh, you know, there's a fall off on this page. You should test something else. And it is built in LLM. So it'll suggest, you know, a new copy to test or whatever else. So it will push suggestions. Most of the time we don't take them. Make them ourselves.
Chase Clymer
Absolutely.
Aaron Zagha
Or we wrote the copy ourselves or whatever else. So you do constantly have to feed it. There's no set it and forget it anywhere in Ecommerce.
Chase Clymer
That’s what I wanted to hear. Thank you so much. Now, Aaron, is there anything I didn't ask you about that you think would resonate with our audience today?
Aaron Zagha
I think the world has talked enough about tariffs and how people are dealing with them.
Chase Clymer
We have a whole episode on it that's going to come out probably next week.
Aaron Zagha
I think one other thing which I'm still kind of astounded by is that people probably still haven't scaled other channels enough and are still shockingly overlying on Meta. And it's interesting. I just got off a call today with another brand who doesn't run TikTok. And yeah, TikTok might be banned. Who knows? I don't know. So then maybe you should be scaled on Pinterest. And generally, you know, I think the reason people haven't scaled is because they're on attribution methods that don't give credit to those other channels or they don't measure them well.
If you run a decent TikTok campaign, I'd be shocked if at the same CPM, it doesn't perform just as well as Meta. If you're watching a video ad, you're watching a video ad, it doesn't really matter what platform you're on. If the CPMs are the same, it should perform the same. You just need to measure it, you know, properly and compare it properly, typically with the GeoLift test.
But given how volatile Facebook's algorithm has been lately, given how, you know, the chance of TikTok getting banned or whatever else. It's really important to be diversified across channels and be ready to throttle one up, throttle one down as one gets banned or CPM spike or the algorithm fails.
Chase Clymer
Absolutely. And you mentioned something there. Can you dumb down what a geolift test is for the listeners?
Aaron Zagha
Yeah. So I'm a huge fan of incrementality testing, really making sure that any ad actually drives incremental orders. And the way to do that is you show part of the country and add, you don't show it to the other part of the country and you see how much revenue lift is.
You have to make sure that the two Geos are comparable. It's called matching markets. So if LA and New York typically have the same number of sales for you and your Shopify, then they're good markets to match. And so you show ads in one, you don't show them in the other. And you see how the revenue changes from one versus the other. And it's really the gold standard.
It's causation, not correlations. A lot of multi-touch marketing or medium mixed modeling attribution is all correlation. And we can get into why that's problematic. But yeah, wherever possible, geolift testing, incrementality testing is the way to go.
Chase Clymer
Absolutely, Aaron. I'm definitely going to hit you up in a couple of months. We're going to deep dive into some of these incrementality debates and really just nerd out on it. But for the folks that are listening, obviously, Newton Baby has some amazing products. If they want to go check them out, what should they do?
Aaron Zagha
NewtonBaby.com.
Chase Clymer
Makes it easy. Aaron, thank you so much.
Aaron Zagha
Very easy.
Chase Clymer
We can't thank our guests enough for coming on the show and sharing their knowledge and journey with us. We've got a lot to think about and potentially add into our own business. You can find all the links in the show notes.
You can subscribe to the newsletter at https://honestecommerce.com/ to get each episode delivered right to your inbox.
If you're enjoying this content, consider leaving a review on iTunes, that really helps us out.
Lastly, if you're a store owner looking for an amazing partner to help get your Shopify store to the next level, reach out to Electric Eye at electriceye.io/connect.
Until next time!
