Guillaume McIntyre: It wasn't really by choice. you know, I graduated at the height of the '08 recession. And so my, my, my number one priority was, to get a job. And at the time, as a business graduates, there were only three main tracks. There was, you would either go into accounting banking or consulting and all, tech firms would only hire engineers.

It was, almost impossible as a non-technical person to work at a tech firm. I did, a lot of interviews for banking, jobs, accounting, jobs, consulting jobs, [00:05:00] but in all my interviews, I never I never felt passionate about the actual jobs, the roles, the people that I met it didn't inspire me. But I still went through the motion and given that, I needed a job. 

But one day I was scrolling through our college career sites and I saw a job post titled "Search Engine Marketing Associate" and I had never heard of that, that, that term before. my apartment's at Berkeley and I'm YouTube being search engine marketing. And I remember super clearly that, there was a, I was watching these video. And this person was describing the difference between SEO and SEM. And I thought, wow. They are truly, businesses who buy ads on Google aid seems to be a pretty big industry. 

And I did more research and I realized that, that at the time, 90 plus percent of Google's revenue was from ads and primarily search ads. And so I thought, Hey, it seems like [00:06:00] pretty interesting. It's a startup. And so I applied for the job. When there, you know, interviewed with the team, felt like the interview process was very different. Not corporate at all. Very like the first interview was actually a group interview with other candidates that were interviewing for the same role.

And the interview exercise was that we had to, they put us into group. And we had to come up with a business idea to pitch our, the interview panel as if they were VCs. And so I thought that was like super fun. And one thing led to another I got the job offer and I just went for it. 

Nima Gardideh: That's such an interesting interview process to begin with. Especially as a younger person going in, I remember doing a group interview with Apple. And I felt it was quite intimidating because it was like 40 of us. How, how big was the group here? 

Guillaume McIntyre: I think it was nine people. So three groups of three all interviewing for the same role. you had to collaborate. But everyone knew that they were going for the same job [laughs]. So it was a bit odd. You worked out and it was a pretty fun [00:07:00] experience.

Nima Gardideh: and so which company was this again?

Guillaume McIntyre: It was Adchemy. Adchemy, at the time was building is search marketing software. And the whole idea was that instead of having marketers manage keywords for paid search, the idea was to like find a better and more efficient way to group these keywords and to do that by managing intent versus keywords.

 And so the company went 

Nima Gardideh: super interesting.

Guillaume McIntyre: the different different cycles. But yeah, it wasn't, a really fun time and, got to work very closely with, PM's with, very talented engineers and data scientists. Yeah, had a great time. 

Nima Gardideh: That's almost a quite early for what is now the standard of how Google operates, which is intent-based around keywords. This is what, 2008 or 2009? 

Guillaume McIntyre: It was 2010. 

Nima Gardideh: 2010, like super early. That's very interesting. So you almost went into like ad tech immediately. so that's, that's a [00:08:00] different path than a lot of marketers take, which is they go into these, like on the brand side. 

Do you feel like that gave you this understanding of how these networks work from the ground up because you were so deep into how they were working? What do you think was positive and maybe some negatives of being on that side first?

Guillaume McIntyre: Yeah. I think that, the positive is that Is that I got to work very closely with, this really bright people my new people that was sitting next to me, or PhD's at Stanford doing machine learning language processing, et cetera and know were very smart sit distinctions because we were building the software that was meant to, convert our group keywords into intense. intent.

It was, it was very, it was a very difficult task. And so as a very young marketer out of college I would do my campaign analysis and but the audience would give me a very, like very candid and sometimes I can a pretty like [00:09:00] harsh feedback, it's flawed or I know this doesn't make sense.

And so it really taught me to be, very thorough in the way I push data into way. I, I present data and in the insight, the insights that I can derive from them to drive actions. So like that, you know, really was a great way to to get started in the field.

One of the, I would say the disadvantages is that working for an ad tech from you, you're not close to the consumer, right? You build, business solutions for businesses, but you you actually have very little interactions with the with the actual consumer which as a marketer that's always something that I wanted to do.

Cause I felt I can do it B2B, it's interesting, but that's not what I was most interested in. When I made move into, you more consumer focused businesses, I had to learn a lot of things. 

Nima Gardideh: Yeah, I think that the want or need, or whatever you want to call it when you're younger of getting it to consumer. Cause that just sounds more [00:10:00] exciting. I relate to a lot. I was more around, I also want it to be close to people's wallets when I first started as a marketer, because you can do all this branding and all this work, but I wasn't sure if what I was doing was coming back and having results.

Would you say that was immediately something you cared about as performance marketing? Cause it seems like that firm was around performance to begin with so were you immediately put into performance marketing mindset from the get-go because of it?

Guillaume McIntyre: Yeah, exactly. We really that's what the whole company was about, was to really enable advertisers to yield better results from the performance campaigns. And so we were using the tool internaly and certainly also to, to run different campaigns. And so performance was always the main objective for everything that we were doing. 

Nima Gardideh: And when you, in this company, the first couple of companies before you started running teams, where they already following some form of like experimentational approach, where they were testing and learning and going through that type of flywheel?

Guillaume McIntyre: [00:11:00] So we would, we would do a lot of testing, but he wasn't we didn't have a clear framework. We would do a lot ad copy testing, then page testing at the time bidding was a big thing. But we never articulated and it never had a great framework to think about experimentation.

Nima Gardideh: When do you think that started becoming part of your job or you're running it or being part of the cultural?

Guillaume McIntyre: So the first time I was really exposed to a very heavy experimentation, experimen focused culture was at Netflix. You know, Netflix is known to be to run experiments on everything they do from like products to marketing and other functions. And that's when I learned how to run experiments the right way.

And to really think about know really formulating your hypothesis super clearly, outlining the KPIs that you want to measure and ultimately, you know, based on the results have a sense of what actions you'll take. 

Nima Gardideh: Yeah, this is one of those [00:12:00] interesting. And I'm super curious how you think about this because obviously anyone here is listening may have read about this concept before. We've got to run experiments, everything should be an experiment. 

What do you think it takes to actually do it right and stick to it? The process of making sure you're truly running experiments in a controlled way. At least in my experience running a growth from the beginning of our work was learning how to like instill this culture in the team.

What have you learned over time on how to do this? Cause obviously I think you're one of the best in the world at this point. I'd love to hear anything you carry with you over the years.

Guillaume McIntyre: I think you have to get everyone who is involved to focus on driving the best business outcome, but not to get attached to like a particular hypothesis. Whether it's theirs or someone else's. And I think that a lot of time is people tend to be they want their project to succeed or they want their idea to succeed.

 but if you remove that and you really [00:13:00] think about, how can we as a team, drive the best business outcome, then you can evaluate hypothesis objectively and you can, and you can stick to your KPIs and actually, drive business outcome because you really, you're not married to any one of the hypothesis.

As simple as that sounds it's actually difficult for a lot of people to be able to detach themselves from, an idea that they came up with a hypothesis and not see that turn out to be true. And having that common understanding that it's all about driving business outcome and, I think no idea is necessarily better than another one. And, sticking to, clear KPIs ahead of time and depending on the results, taking a play action but not making excuses that, oh, you know, like maybe the results didn't work out because of, X, Y, and Z. And so like those other things that would that I think ultimately tries to like an experimentation focused culture. 

Nima Gardideh: Yeah. It's interesting. Cause you, and I don't know about you, but at least for me, a lot of how I ideate experiments are based on intuition. So [00:14:00] organically because it's coming from your intuition, you're attached to it because it feels like it feels right that it's going to work. So this part is extremely challenging for everyone.

I think I would say I'm still learning to some extent. But that's probably like the most important part to me as well. It's this disconnection, this association from the ideas itself themselves When you are a Netflix was a scale big enough that you were running experiments with no issues. Like have you been in a, scenario where there's not enough data to run clear A/B tests?

How do you instill the same overall approach when data is not as available as let's say a company the size of Netflix?

Guillaume McIntyre: So when there's not enough data what I remind my, myself and the team is that, if you want to new things and with high statistical significance, we have to test big ideas, right? Because if it has very small ideas and there's no data, you'll never feel confident that what you learned is, is actually indicative of real performance. You [00:15:00] have to think much bigger. 

And the other challenge is that sometimes if there's, if you cannot gather enough, data is just not worth testing, because then you're just going through the motions for the sake of going through the motion. But if the data is not reliable or you can't collect enough data, then that's when you really have to use your intuition and just hope that you're making the right call.

And if you're not making the right call and you realize that later, and then you can course correct and do that. 

Nima Gardideh: Yeah, that part. I think that, and this is probably one of the hardest things in startup culture is in the beginning, you just don't have the data, so you have to do this intuitive path, but then at some point you have to shift your culture from running these intuition based tests to this experimentation.

It's have you gone through that shift in a company? Cause I think you were pretty early in Instacart, but I don't know if you were, they were already running tests and things like that.

Guillaume McIntyre: At Instacart, we had a good amount of, data in just because the scale at which we operate in was large. But they're, are areas [00:16:00] where we either didn't have enough data. We couldn't collect enough data. So as an example, we know when we were running affiliate marketing. Generally the most common affiliates for an affiliate marketing program for performance is, loyalty sites, coupon sites like those are usually like, very large buckets for an, the affiliate marketing program for performance teams. one of the things that I always wondered is, how incremental is or some of those loyalty sites coupons.com and others. And it was really bothering me that we were spending money on, on, on some of those partners. And we couldn't actually identify whether those new users that were being attributed to those loyalty partners were in fact incremental. And so we thought about ways to measure. And the reality is that, you know, we weren't big enough of a player that we could strike, very close partnerships with those [00:17:00] publishers to run incrementality experiments. and so we know we didn't bother trying to measure that, we just said, Hey, you let's make sure that the CAC is low enough. So that UMD incrementality is also very low, then your backs out to something that's reasonable. those are some of the times why we, it wasn't the ideal decision. But we just did it. So we wouldn't be wasting time.

Nima Gardideh: Yeah these, become tough as this. It's quite nuanced because you want the growth, but there's also like the downside of you pull it out and you're wrong about this, and you could lose the momentum on that channel. 

I wanna step back a little bit before I actually want to go deeply into your Instacart experience, but just to touch on this experimentation approach again, do you feel like there it's become the de facto standard and it's something that becomes a de facto standard?

I always wonder if, is it the best model? Obviously I think both of us have been in teams where we continue running it. So I'm super curious if you were to try to be a devil's advocate. Do you think there is a different approach to your marketing culture [00:18:00] that is more successful potentially that we're missing out on right now?

Guillaume McIntyre: When you're so focused on running experiments and sticking to this framework very closely. You can lose out on the bigger picture. And what I mean by that is at Instacart we would run lots of experiments, lots of experiments across every single channel. And at times, we would close the experiments and get learnings. They would celebrate, oh, like we, we validated the hypothesis . 

And one of my mentors, one time told me that Hey, it's great. You ran these experiments. You got the results you wanted, but that took so much energy. Was it really worth. To prove that these $10,000 was actually not incremental to the business, or could you have, simplified could you have not run this? The analysis, find a way to save that $10,000, but then, focus your energy on something else. 

And that, to me was really like a, like the light bulb, went [00:19:00] on. And I, I realized that while we've spent so much time experimenting across everything from paid social to paid search, to programmatic, to TV every single channel we had, very rigid experiments driven approach and, in a very rigid measurement approach.

But sometimes, like thinking back, for some of those, experiments, it wasn't. And I think we could have driven, even more business outcome. How do we know simplified? And just going with my intuition. 

Nima Gardideh: That's interesting. Cause so you're saying the rigidness of ensuring that every dollar is accounted for and the best way possible almost is getting into way of potential creativity put behind better experiments. So still within maybe the experimentation framework, but being so focused on incrementality and every dollar being accounted for it has its negative connotations, essentially.

Guillaume McIntyre: Correct. And [00:20:00] so as an example you could run two ads. One of them that looks very well-polished that feels on brand And you could run another ad that maybe looks bad, but that's like super, CTA driven and that you feel could deliver a good performance and you could A/B test those two ads and try to, they felt like which direction should we take? But the reality is that, as a marketer, you shouldn't run an ad that looks bad even if he drives, great performance, right? You should try to make the ad that looks good. That's on brand and make that drive, great performance. I've seen performance marketers that, we will want to A/B test everything that they really lose, they really forget that at the end of the day, it's not about running experiments about figuring out the best way to drive to, to drive business growth. 

Nima Gardideh: It's almost, there is a there's this joke I love hearing, but there was this partner at YC that gave us this advice a while back his name's Kevin Hale. He used to say that if you just continue A/B testing forever, you're [00:21:00] eventually going to become a porn company [laughing]. Um, and I think this is like quite, truthful because you can definitely find ways to sell something or make it extremely performance driven.

And those do not necessarily go hand to hand with building a great brand, something that will last a long and be iconic in the world. So that, that example is quite insightful. I think because a lot of people we've hired from past performance marketing experience have this.

Unfortunate, but real habit of thinking that design doesn't matter, it's just about the copy and, and, and the approach. But it really does to us, and we have a very strong creative team because of it. So that, I think we're very aligned on this and caring too much about every dollar. I think it's a good general approach that, because some people just don't have that. And I think if you do have that, that's, the sort of potential negative artists you're blocking your creativity.

 So let me go back to to Instacart before we get into it, I want you to ground us, cause I actually don't know, give us a story of when [00:22:00] was the state, what was the state of Instacart when you joined? And what were the challenges that you, you were sort of presented with as, as a marketer? And I don't remember, I don't know if you were immediately VP or you had a different path to becoming the leader there. So I'd love get a grounding on how you started your journey.

Guillaume McIntyre: Sure. So I so I joined Instacart in 2017. The company was founded in 2012 and for the first five years at Instacart, Instacart grew almost entirely through word of mouth, the business model and the product market fit was very strong. And so in had no investments in, in marketing. And so when I joined, you know, it was right after Instacart had to raise a large round of fundraising. Back then, I think it was, $300 million mostly from Sequoia, which $300 million today you know, five years later doesn't seem that big [00:23:00] because a lot of companies are raising hundreds of millions, but back then was like, wow, like the huge round of funding. And so with a large amount of funding, calms a lot of pressure to grow fast.

 And so, as you know, I'm sure, organic growth can only take you so far and at some points you want to market your product to really accelerate the growth. And and so th the challenge was, can Instacarts leverage paid marketing to grow faster. And so I was hired to lead all digital marketing and to really to see if we could to see if Instacart could actually use paid marketing to accelerate in store. And what was interesting is that, you asking me, you what was it like an Instacart I remember, like it was yesterday that I would tell my friends, Hey, you know, I just joined this company Instacart, in San Francisco. Oh, cool. You know, I love Instacart. 

And then I would drive down to the pennisula and meet some friends and say, Hey, I just joined a company called Instacart. And no one knew what Instacart was. And he [00:24:00] wasn't in the peninsula, 50 miles south of San Francisco. And so I realized that, wow, you know, Instacart is really a big just in San Francisco.

And of course I'm like there where other places where there's like in a big know, like in New York and in other urban areas, but even, like in the peninsula, like no one knew of Instacart. And so I thought, wow brand brand-awareness is really low. And two screens to take a lot of work to actually try to like, grow through marketing.

 that was my first project. It was, can I prove out, can I improve to the exec team that investing in marketing is the right thing to do? you know, so I started off testing very in a bottom of the funnel channels, like paid search and paid social.

And once, we got some wins then, expanded to other channels like affiliates direct mail display. And then we moved up the funnel and then over time we were able to show in a wins across the funnel and across different channels. And that was overseeing both the demand side of marketing [00:25:00] as well as the supply side. 

Nima Gardideh: Oh, interesting. So I would definitely want to get into the supply as well, but let's start with the paid search, paid social to the next iteration. So the next set up channels, like how did you, what is the inflection point in your opinion, if you were trying to like generalize this idea of diversification of channels, when do you think is the right time for people to take these two channels that are, that become the de facto standard original channels you start off of as paid search and paid social. When do you expand beyond?

Guillaume McIntyre: When your team is small and you have limited resources, you ought to pick the channels that you feel are most likely going to work for you. And for the majority of companies in, for us as I Instacart new, I felt like paid search and paid social where like the channels that, that could that, that were most likely to work out for us. 

We started launching some campaigns and I would say that, we didn't start expanding into into other channels until we saw [00:26:00] wins in those first two channels. And if you think about it, and my boss at the time was the Chief Growth Officer and, one of the advice he gave me was let's show wins on a couple of channels, So then, we can get people excited about extending marketing experimentation in driving growth through marketing. But first we have to show that it can actually work for a couple of channels. and keeping in mind that Instacart had never invested in marketing. And so for them to start, seeing, pretty large investors in marketing for the first time, was, a bit uncomfortable naturally. 

So I would say that, once you can show some wins on your channels and once you can show some growth, that's impactful, then you can start looking at other channels unless you feel that you've picked the wrong first two channels, then you know, if you feel that, Hey, our business is not set up to be successful on paid search because no one is searching for the [00:27:00] things that we sell then, pivoting to a different channel early would make sense.

Nima Gardideh: And so you put it basically, if I were to try to understand, you're saying basically it's a combination of showing, proving that the current ones work before you move on to the other ones are proving that they don't work and you move on to the other ones? 

Exactly. 

Yeah and then resources. Can you actually expand to the other channels realistically with how many people you have or how much capital you have... 

Guillaume McIntyre: yeah. 

Nima Gardideh: To do it? Would you say that you're not trying to maximize the channels then? Are you diversifying quite early instead of trying to get all that you can get out of the original channels or do you think it's like they go hand in hand, you start doing all of it.

Guillaume McIntyre: Yeah, So it depends, I would say that, if the goal is to grow as fast as possible then you need to figure out with the time that you have working against the most growth with the same efficiency. And, growing a Facebook [00:28:00] program from, spending $10,000 a week to a million is going to get much more difficult when you get in the hundreds of thousands of dollars. 

And so so my push would be like, if you find, go really fast then if you identify that there are low hanging fruits across other channels that you can capture pretty quickly. So try like affiliates or display or YouTube or another channel, then I would diversify early on.

Nima Gardideh: Yeah, that, that makes a lot of sense. I think this is, it's a fact of how fast things are moving. And a lot of, if you're venture backed that you probably, in this situation of, Hey, you got to move on quite fast in order to match the growth rate that your company wants to achieve. So that sounds super familiar with a lot of our clients. Where they come to us and my first response is quite often, like you can get more out of these channels. Like, why are you caring about diversification so early and quite often, the answer is [00:29:00] what you just talked about. 

Although I do find that some marketers aren't willing to they think they've hit the ceiling quite early with some of these channels because they hear exactly what you mentioned, where, okay it's, easy to spend, $200,000, 300,000 on Facebook, but if you want to do a million a week it gets harder, but I don't think people understand how hard it gets and how fast to get, it gets harder. So there's still more, but if you have the right process, you can still continue on locking a bunch of growth there, right?

Guillaume McIntyre: Exactly. So there's always, more you can do. But you also have to balance like the expertise on your team, having a Facebook marketer that can drive a program from zero to $200,000 a week may not necessarily have the skills to go from $200k to a million, right?

Because then you need to be much more granular in your approach, need to experiment faster. you'll likely want to leverage internal data to distill meaningful insights to, to guide your experimentation. And so those are, that's another thing that you have to consider. 

Nima Gardideh: And before, I definitely want [00:30:00] to get an understanding how you were thinking about supply versus demand, but before we close this part on resourcing, I know you definitely hired agencies in the past. Like, what is the heuristic you use for deciding to work with partners to help expand the resourcing and the experimentational speed.

Are you always going to work with agencies? Do you think there are stages of companies that make sense to work with external teams, paid teams, or how does it work in your mind?

Guillaume McIntyre: To me, if you are leading performance on growth, are they at a super high growth company? You must have an agency that can move as fast as your internal team. So that's, it basically requirements. the second piece is that you want to, you know, an agency that also thinks the way you think.

If your internal culture is experiment, experiments, you want an agency that can, they can do that. so you can be on the same wavelength. and third, I would say that it's agency, that's [00:31:00] also highly adaptable because any high growth company will we'll have different business objectives then it will across a different period of time. and so if you have an agency that cannot adapt to like, uh, new CAC targets or in events to optimize against, or different audiences to go after, then it's gonna make things very. 

Nima Gardideh: yeah, I think that we talk a lot about this, of the culture fit between our team and the teams that we've worked with is super import. I'm glad also the people on the other side think the same, right? If it, if we are experimenting faster than your whole organization is experimenting, it doesn't work.

And vice versa. We almost need to be on the same type of rhythm with these types of things, especially there's also agencies that just don't do experiments as tightly as maybe internal teams do. So that's also a problem. I've definitely seen it or run into a lot of these problems in the past couple of years of trying to help brands.

what do you think about I feel like, and correct me if I'm [00:32:00] wrong, but Netflix has a lot of items on the system, but the supply is fundamentally different. So you're not going to, as a marketer do supply, but it has in my head, some like marketplace dynamics where there is demand and supply. Instacart obviously has that are there, and maybe the answer could simply be no.

Are there any similarities for you when you were thinking about. the sheer amount of skews and items that Instacart had and leveraging that for growth. And across the board, any of the information that you were at similar sort of thing, where there's a lot of items is there things that you carry that you think every time I walk into a company with a lot of items in a marketplace, I'm going to think about it in this style, in this way, or maybe the answer is no, and it's very unique to each company, but I'd love to see if there are generalizations for you.

Guillaume McIntyre: So th what I think about when then when I walk into a new company is what do we have that is unique and that will give us a competitive advantage [00:33:00] from a marketing side? Instacart is the fact that Instacart was partners with so many different retailers. And as a result, anything that you want from anyone at those retailers, Instacart has.

Instacart has incredible breadth. we have things that people really love and they can't get elsewhere. from a marketing standpoint that's what, those are the things that we can lean on and to get people, intrigued, excited and get people to transact.

The information it's, it's similar as that, you know, the information, there's a report on every single tech news, but it will break the biggest tech news, and so it's leveraging those moments to generate awareness for the brand and ultimately subscription growth and revenue growth. 

Nima Gardideh: So if I were to just try to understand, so you're saying basically the unique value within the market, marketplace or the dynamics of the business is what actually matters. Not so much the sheer volume or things like that is what is it that people get [00:34:00] excited about or end up converting because of that they can leverage. 

Guillaume McIntyre: Exactly. Yep. 

Nima Gardideh: Yeah. So for, at Instacart, the, were there any points where you're you had supply constraints where it was just not enough supply in an area? Like how do you think about what are those, was there a metric that you looked at for utilization or health of a specific market, or even like a neighborhood? How did you guys think about that?

Guillaume McIntyre: Yeah. So we went through, the different cycles. And initially it was all about, we had healthy supply, so it was all about generating more demand and and just getting more consumers excited about Instacart and shopping for their groceries on Instacart. And what was interesting is that as we generated a lot of demand then would became supply constraints.

And so then, the focus was how do we get more more shoppers to pick up shifts and deliver groceries. For our members. And that was at a time when Uber, Lyft and almost, so many companies that were [00:35:00] providing on demand services, everyone was competing for the same talents and for on demand contractors.

So that was pretty challenging. And we always had to balance between the two, and sometimes we had to pull back on, on demand to really focus on supply and vice versa. And so I think that unless you become in your growth phase it's very difficult to have a perfectly balanced marketplace, right?

Because it's hard to predict demand. When you're growing so fast and you don't have enough data to really predict things accurately. So you just have to be comfortable with with the balancing between the two and be more sometimes would be more reactive. 

Nima Gardideh: Was there sort of like metric of some sort that you're looking at to decide when to switch over? Like, Did you figure out, oh, if there's not enough, if the maybe delivery time is higher than this than we are clearly, this is an unhealthy part of the marketplace that we should spend more on the supply of. 

Guillaume McIntyre: Yeah. So I forgot [00:36:00] the, the exact metrics that we were looking at but at a high level, what we wanted to make sure of is that if you acquire new customers, that these customers have delivery availabilities that that can meet their needs. And for Instacart, it was all about, it is all about very fast delivery in one hour or less.

So we wanted to make sure that, Hey, if Nima, if you come to Instacart, Whether you are in Brooklyn or in Burlingame that you can't get your groceries delivered to you within one hour, not only for your first purchase, but also for your next purchase and the one after that. And so when we, saw that, Hey, we actually cannot deliver groceries to Nima within one hour. That's why we focus on, on, on improving the supply so we can continue to deliver that great user experience. 

Nima Gardideh: And how, what was the granularity that you were looking at these? Was it just citywide neighborhoods? Like how did that work? Were you just looking on markets essentially or going deeper?[00:37:00] 

Guillaume McIntyre: it was pretty granular. And it's, it was less about from what I recall, it was less about like cities or zip codes. It was more about particular geo coverage. Some cities are very, very large like LA, and others are very much small like San Francisco.

So it was more about, based on the way we know we are drawing out the map, do we have enough coverage in those areas? So we didn't have a a cookie cutter approach of saying, Hey, this is, there's just to get cities. It was more about we draw our own map and then optimize based on that. 

Nima Gardideh: Is that how also the ops team was functioning, where there was like different ops teams per these groupings?

Guillaume McIntyre: Yeah. That I am. I'm not, I wasn't, I'm not too familiar with. But I think it's yeah I'm not sure. 

Nima Gardideh: Yeah. So maybe I can ask you this other question. It's just the structure of the team, because I've definitely seen marketplaces where the VP of Marketing is only in charge of demand, for instance. So, What is your belief, do you think you should, as a marketer. [00:38:00] They're looking at both sides of these things, even if one is a totally different approach, which the supply and correct me if I'm wrong, I guess it's quite similar Instacart where you had people, you didn't have to actually close businesses.

At least in the beginning, it was a lot of people delivering manually. Do you think in marketplaces, people should centralize the growth across both sides of the marketplace. So then someone is looking at it across the board, or do you think it makes sense to have maybe like a VP of demand and the VP of supply, almost caring about this problems? Cause they're slightly different.

Guillaume McIntyre: You know, There are pros and cons for each approach, but ultimately you want people to align on driving the same objective, which is, business growth. And, if you're working on the demand side and you realize that the supply side is constraint, then that you won't be able to drive business growth, right?

So then you will naturally I pull back, right? I think if you have people who are, focused on just their [00:39:00] own individual objective and are not highly aligned with, the other side, then the creates many inefficiencies and you can't drive growth together. So to me, it's less about the org structure, but it's more about instilling this fundamental belief that, whether you on the supply side or on the demand side your job is to drive business growth. And so if, if the revenue growth is not there, then you know, no one wins. 

Nima Gardideh: So it's almost cultural alignment on trying to help the business success succeed as much more important than how the team structured. 

I want to get into we a little bit, cause you had like, I guess a one-year stint at The Information, and then you went back to more groceries. Super curious, how you, how you thought about this. Do you think that you felt like you had already built this muscle and you wanted to now do it even better this time? What was the personal reason you, you went back into, the similar industry although slightly differentT

Guillaume McIntyre: yeah. You know, When I I left Instacart, I was pretty [00:40:00] satisfied with what I had done and it was, ready find new chapter, which is why I joined The Information. And I really enjoyed my time there. And I had no interest, so I'm going back to the grocery industry.

But I was introduced by the founder by a very good friend of mine and, one thing led to another. But what got me excited about the businesses that, as I mentioned earlier, I am French and I grew up in Paris. I enjoy a lot of French foods. And when I go to the supermarket even in places like San Francisco, where we are fortunate to have, more access to, to, to a diversity of groceries almost every single supermarket in San Francisco, whether it's whole foods or Sprouts or Rainbow Grocery, or, all of them almost have the exact same French food. 

And if you go to the French aisle or the international aisle, you look at the French section, it's typically french baugette that doesn't taste like [00:41:00] french baugettes, or there's check your tree. That's actually mostly like Italian check between. And then you have, a pastries that you know, a very, very Americanized pastries. I, I just while I was never satisfied with my, my local groceries. I just felt like that's just how the school and, I'm in America. I shouldn't expect to have all these things as if I was in France,.

But then what I realized, what we was doing and what they started doing in, in, in the uh, several years ago is that they start catering to the Chinese community. And they basically did to the Chinese community, what I was hoping, what, whole foods and all eithers could do for me as, as a French person. we was, was sourcing the foods that people like loved from China, the foods like different spices, different vegetables foods that people in the US food foods that Chinese people in the US had forgotten [00:42:00] about because, they had never seen them for decades. and that's how, you know, that's why, you know, people really loved Weee!, right? Because we brought them to food that they loved and brought them the foods in the, from their hometowns or things that, that, they grew up eating. I thought that this model really resonated with me.

And and I didn't know that, maybe naively, I didn't know that, the pain that I had trying to shuffle for French groceries is also the same pain that Chinese people have that, that Japanese people have, that Mexicans have, that Koreans have. And that problem that we solving I think is very unique.

And the product market fit is great. I, I enjoy, building things. And also, I look forward to the day one, everyone knows about Weee! the way everyone knows about Instacart. And that gives me, excitements. And I think, it's going to be, it's not going to be a walk in the park, but I'm sure we can do it. And so that's why I went for it. 

Nima Gardideh: Yeah. That's super interesting because I resonate to what you're talking about. Cause I'm Iranian and they rarely can, I get what I want and New [00:43:00] York and I, and I wonder if you guys have actually solved this for me and I should look into Weee! or buying Iranian food as well. Cause I go to Toronto, my parents have found all these like small shops that have what they need to cook a proper Iranian food, but it's not easy in New York at all.

And it's quite interesting. So walk me through, we, you joined much earlier in its cycle right then and Instacart now. So what do you think is different trying to solve a similar problem, but it sounds like it's quite different and in a way, in terms of who you're going after and what the supply is like, are you approaching things from ground up thinking about everything from scratch again?

Or do you feel like you're applying some of the things that you carried from Instacart? Knowing that okay search was a big channel for you there. So obviously it's going to work for Weee!, How are you thinking about that? Is there I'm curious about your process, essentially as a marketer, you tried to look at things fundamental, fundamentals every time, or do you grab heuristics and apply them very quickly at the beginning?

Guillaume McIntyre: [00:44:00] What's different about we is though in a, Weee! were vertically integrated and we own our inventory. you know, while Instacart, you though, we had to be mindful of that supply side, making sure that we had enough shoppers to actually deliver orders. At Weee! we have to be mindful that when we have super hot items sometimes these items can sell out, right?

So like, how do we, if we want to focus on, on, on exclusive and hot items as part of our, as part of our marketing strategy, how do we take into account that we don't have an unlimited supply and how do we manage that? But one of the things that that I've applied, at Weee! that I learned,at Instacart the same at Netflix is that when your brand has very low brand awareness, no one cares about your brand. And even when your brand becomes, has good brand awareness, there's still a lot of people actually don't care about your brand.

What they care about is the things that you can, that they can, that you can deliver to them, [00:45:00] that they can purchase from you. At Insacart people didn't care about Instacart, the Instacart brand early on, they just cared that, oh, I can get my groceries from Costco delivered through Instacart. And that's what got them really excited. 

And at Weee! It's the same thing. Like people I don't care about the Weee! Brand today. Like what did, what they really care about is that we deliver them this pink pineapple that comes from Costa Rica, and they've never seen a pink pineapple in their life and tastes, better than I think better than the gold pineapple. And so those are the things that, that matter.

And so when we, in our marketing strategies, it's always about, why can't we, what do we have that will get people excited that people will find delicious and the people will love, but it's never about, Hey, you know, where we, we delivered groceries where like a cool brand check us out. It's never about that. It's about, what can we do for our customers and how can we delight them through our product offerings. 

Nima Gardideh: It's almost as if you're saying basically direct marketing and that's the whole philosophy of direct marketing is, Hey, here's the [00:46:00] solution to your problem. That is much more important than the ethos behind the brand and what it stands for in the beginning, because you still want to solve the problme. The people in this scenario, maybe didn't even know there was a solution for, so it's gotta be quite fun to discover some of these pieces of grocery on Weee!. 

I guess this is the interesting part to get into, cause I know you're obviously a good you're quite good at thinking about incrementality. I remember the first time we had a conversation, you were probably one of five marketers that year that asked me about incrementality. Even though we are quite into it. So maybe tell me about when you realize that you have to think about it this way, because performance marketing, you obviously think that your numbers.

Correct. If you look at them, you're showing these ads to people convert, then you feel happy because you have $200 CAC or whatever. But then you realize that the next dollar is not going to be the same at the same efficiency. So walk me through your journey of like, when you learn about incrementality and now how you thinking about a brand being part of incrementality, because in my [00:47:00] experience, at least when you do brand spend, you are still spending in a positive direction quite often.

So those are the two steps. Like how did you think, how do you, what do you think about incrementality at like the performance layer? And then how much do you think about it at brand marketing layer? Which I assume you got at Instacart.

Guillaume McIntyre: Yeah. So for the performance standpoint, I, I don't recall, when I was first exposed to two incrementality, I think let's maybe in the, 2015 I actually, I remember the very first time I was exposed to it. there was a professor at Berkeley who conducted the study.

So he was, I think he was also working at eBay and he conducted a study to measure the incrementality of paid search. And then what he found is that the best practice paid search marketers of bidding on paid search brand high, extremely low and maybe even, almost at zero incrementality and in his [00:48:00] paper that was published was, I think the first paper on marketing incrementality and what he showed is that, if someone's searching for for eBay, whether you have a paid search ad about eBay or not, people will come to eBay. So, you know, While you were capturing that click and you're incurring costs you actually wasting money. 

I found this, his paper is super interesting. And so I emailed in we got, we actually, I think we might, at a German bakery in Palo Alto. And I talked to him about his paper is like, wow. I mean, I'm glad, thanks for reading my paper. It was, pretty controversial, at the time. And that really got me thinking that they're going to be, to realize that much as marketers think that the ads cause business growth, a lot of times the ads don't right.

It's all about, like this, this attribution problem. And and so as I got more interested in a topic, I talked to different people in the space and to learn more. and that really, made me realize [00:49:00] that marketers should be focused on solving business problems and not marketing problems, because, you can solve their marketing problem to have it really low CAC, but then you may realize that your low CAC actually drives zero business growth. And it's not incremental to the business. From a brand stand point, I recognize that brand has to be incremental. I have in a different ways to evaluate the impact of brand, because I think they, if you use the same type of criteria to measure Brent performance you will never invest on brand because you will never get the same type of of direct, of, you'll never be able to read the results as quickly.

And so for brand, it's more of a longer term strategy. and measuring, things on a multi or on a quarterly basis to try to quantify the impact of your brand know.

Nima Gardideh: so actually, first of all, I recognize that maybe we should define incrementality. So I'd love you to define it a little bit more succinctly because I think that example for a brand that search [00:50:00] is very clear, but it can obviously get a little bit more complicated than that. And the second question I have for you is when do you start running incrementality test? Is it from the get-go is at a certain scale. How do you think about that?

Guillaume McIntyre: THe way I describe incrementality is, today, Nima you see an ad about Weee! that tells you that we have the best Asian groceries. We don't have, Iranian questions yet, when we do out, we'll let you know. So you see an ad about, we know you, you, you engage with the ad, you visit our sites but you don't make a purchase. And then a few hours later I send you a message as said, Hey, Nina, I just joined Weee! you should check it out. W e deliver to Brooklyn, it's a great service. Tell me what you think. And you're like, okay. then you go on the week and you you place a purchase. 

So incrementality is basically a determinate is trying to determine what caused Nima to actually to make that [00:51:00] peurchase. Was the ad, actually, did the I've contributed to you making that purchase or was it actually just me telling you check out Weee! and as a results, you checked out Weee! So if it was, if I was the reason why you made a purchase then the ad had no incremental benefits to you making that purchase.

Nima Gardideh: And now walk me through, when do you start doing this? Is it at a certain scale from the beginning? And how often are you looking at this data? And I'd love to see actually, because we do it in a certain way, and I'd love to know how you guys are looking at it. Are you looking at the sort of on, on platform data?

Are you, do you have your own attribution models and building incrementality on top of those? What are the different approaches that you maybe learned? 

Guillaume McIntyre: Yeah. You really wanted to start thinking about incrementality, once you budget gets large enough where it's at, you can actually run those types of experiments. if you are managing in a small budgets and the only thing you're doing is [00:52:00] Facebook, then you can, generally you can get a sense of how much incremental value of Facebook ads contributing to your business.

But if you have Facebook running and then Google and then YouTube and and TV, then it's very hard to tell, which channel is actually driving. What and then on top of that, you have PR you have, strong word of mouth. And so that's when you want to have a better understanding. And so in, put it another way.

I would say that once you, once you're unsure what's actually driving growth is when, and your growth is significant, is when you should start thinking about incrementality. 

Nima Gardideh: yeah, it's interesting. Cause I've seen people even do it within one channel when it gets large enough where they're like how much should I expect this channel to drive? More when I increase the budget. So almost like the question is where should the next dollar go? Or should I increase the yeah, commitment to the specific channel?

So yeah, that, that resonates a lot. And do you [00:53:00] think you have to hire a data scientist at this point when you're running a paid growth at a certain scale, because obviously this is maybe beyond what a lot of marketers expected to have to do is care about these modeling approaches of thinking about cash.

Do you do this yourself? Do you have a data science resources? Do you outsource it to someone else? Like how do you think about that?

Guillaume McIntyre: Yeah, so you definitely need data science resources to do that. Well, If you're the fortunate to have a team of marketers and I have, in the past who can actually, run some of those analysis and some of those models, within your team, that's great.

But having a strong relationship with your data team is very important so that you can, they're really aligned on, what's the best way to measure a specific channel the type of experiments that you want to conduct. and having an experience data person is very important. 

Nima Gardideh: yeah, exactly. I found it hard that if you hand this problem to a data person that doesn't miss really have a [00:54:00] great background in attribution already and understands like some of these concepts it will take them like way longer to crack it than someone who understands the area. 

Guillaume McIntyre: Yup. Yup.

Nima Gardideh: So we have one question from the group here. I'd love to hear your thoughts is, "when is a good time for a startup to hire our first performance marketer?" You' certainly join in when the need was clear, but I wonder what other than, Hey, we need growth and we need it faster. What is a good time to explore these channels?

Do you think that at the beginning it should be all attempting to get to word of mouth or do you think paid can be used as the first channel to even prove out a product or market.

Guillaume McIntyre: So it depends on the business model. You know, I know that there are, there are some, B2C companies that have been built from day one through performance marketing. But generally I would say that, for most businesses I would highly recommend for founders to get to probably market [00:55:00] fit without any paid marketing. Because, if you, if your business cannot grow with any paid marketing, It generally means that to product market fit is not there. 

And so whether, you try to hack your way into it. It's usually not sustainable. And so would really focus on, identifying a problem that you want to solve it. Show, show that there's an audience for it. And then once you figure that out, then you can tell the world that, you have a solution that people love. 

Nima Gardideh: Yeah, it's almost like you want to think of it as fuel on the fire, but not to start a 

Guillaume McIntyre: Exactly. Exactly. You should never view performance marketing or any type of marketing as, domain, no fuel source. It should just be, it should help you amplify your, your, your growth.

Nima Gardideh: yeah. And another question that came up here I think it's actually a very good clarifying question about the earlier conversation we had about how to decide to run A/B tests or not in the early stages of a company. The question is, "would you still run an A/B test, but [00:56:00] lower your threshold for the P value?" so, Your significance tests are looser or do you think you should just not run an A/B test and just stick with your intuition?

Guillaume McIntyre: So it depends on what your test. There are things that you should always test even really early on, like running ad copy test. You should always have, multiple, ad copies wrote rotating it and to tell me which one is best. It's very easy to do. For things like landing page test in a way typically requires an engineering supports.

I would think about, if you have a limited engineering resources is it worth running a landing page test, whether your P value will be high. and so I, at the end of the experience at the end of the experiments, if you're not sure whether at any pages I actually performed better or not, is that worth more than an engineer building an additional feature for your product? that's why would I balance things. Yeah, that's how we think about it.

Nima Gardideh: Yeah, I think that takes me [00:57:00] to maybe the last area we'll explore which is obviously when you're doing performance marketing or general growth marketing, your ultimate goal is as you put it business value. So you're not just bringing people in the top of the funnel.

You care about how deep they go in if they purchase and if they come back and you care deeply about the value of the customers that come in, how do you, and maybe the question is twofold. One is, do you think it matters what the organizational structure is such that you're involved in the product to some extent?

And the second part is how do you, I guess, build a, the right relationship with, the user experience, such that you can make sure, first of all, that you have access to the data, but also are running experiments. On that part of the funnel as well, not just the top of the funnel. So you're going in and running experiments across the board.

Guillaume McIntyre: Can you repeat that question one more time?

Nima Gardideh: Question one is I guess the way I put it as does, do you think that organizational structure matters right? That I've seen a lot [00:58:00] of VPs of Growth, our chief growth officer that did not have a whole engineering and product organization underneath them. But uh, you also see VPs of Marketing that don't necessarily have any resources or a say in priorities.

So that's more of a lead through influence situation. So I'm super curious if you think that actually matters. And then the second part is how do you combine, do you combine your experiments across the board or do you think there's siloed?

Guillaume McIntyre: So what I would say is that um, ideally your product. And your marketing org roll up to the same person, right? Whether that's that person is the CEO or whether that person is an exact working on the growth as a whole because if you have that, then you can no, you can really align the objectives of both the teams, right? So, so then, you know, as, you know, if you are, leading those two teams as the growth, because as you can see that the marketing has some really good hypothesis that can [00:59:00] catapult the business growth so that you can have your protein prioritize and vice versa, right?

If the product team is like pushing some really exciting features that you can deprioritize some things on the, in the marketing queue. I think that if you don't have you doing really close alignments And the other way to approach is that the marketing exec and the product exec. cAuse I have to work very closely together and have to be highly collaborative.

 Today we have a very close partnership with my counterparts on the product side. So that, you know, she and I can try things based on what's best for the business. And so like that is very important to be able to drive fast growth for the company. 

Nima Gardideh: So in your mind, like the most ideal scenario is pitting the experiments against each other, across both marketing and products. You're choosing the highest potential ROI things across the board, as opposed to maybe dividing resources.

Guillaume McIntyre: Yeah. Ideally. Yeah. yeah. 

Nima Gardideh: Totally makes sense to me. I think this is something we, we uh, we talk about with our clients a [01:00:00] lot and some companies are quite better at it because the culture is set up correctly in that front.

And some have, there's politics involved in we're part of it quite often because we'll go in and try to explain why this experiments is important. So it can get prioritized. You're super familiar with, when you talk about partnerships between product counterpart in the org.

 Cool. So I think that's all, I've got it. I know we, we, we went basically the full amount, so thank you so much for doing this with me. It was super fun to get to know your story and talk about it. And for those listening, thank you for sticking out through this whole thing. And hopefully it was useful.

You're going to get an email about giving us feedback on this as a totally new format for me. So I'm looking for feedback and be as descriptive as you want. And hopefully not too mean [laughing] about, about 

 [MUSIC FADES IN] 

Thanks so much everyone Yeah, we're going to do these every few weeks and hope to see you again. And thank you so much for doing this with me. 

Guillaume McIntyre: Great thank you Nima. 

 [Mic bleed] 

Nima Gardideh: Thanks for listening to our show. Get our episodes. As soon as they're [01:01:00] released, just tap that follow or subscribe button, wherever you get your podcasts.

Plus if you want to join our live discussions where you can ask us questions, as we record, sign up at pearmill.com/hypergrowth-podcast, We'll see you on the next episode, on The Hypergrowth Experience.