Why Retailers Can't Ignore Competitive Intelligence With ClearDemand CPO Rajat Nigam
In this deep-dive Omni Talk Spotlight, Chris Walton and Anne Mezzenga are joined by Rajat Nigam, Chief Product Officer at ClearDemand, to unpack the critical role of competitive intelligence in modern retail pricing. Formerly leading pricing innovation at Amazon, Rajat explains how engineering, data quality, and normalization are central to making pricing intelligence truly actionable — and profitable. From assortment gap analysis to real-time pricing enabled by electronic shelf labels, this episode explores the full retail stack of competitive data strategy.
Key Moments:
- (1:00) Rajat's background and Amazon's secret pricing project
- (2:40) What makes “good” competitive intelligence
- (5:00) The three hardest problems in competitive data
- (7:30) Matching pack sizes and normalizing prices
- (10:00) Competitive data beyond pricing – assortment, promotions, planning
- (14:00) Where comp intel works best: grocery vs. luxury
- (17:00) Role of price elasticity in retail strategy
- (20:00) How to start: KVI items, category focus, and data gap analysis
- (23:30) Agility in pricing with ESLs and real-time data
- (25:00) Final thoughts and predictions for the future of pricing
#RetailTech #CompetitiveIntelligence #PricingStrategy #RetailInnovation #ClearDemand #OmniTalk #GroceryTech #RetailData #amazon #PriceOptimization
Music by hooksounds.com
*Sponsored Content*
00:00 - Untitled
00:20 - Introduction to Pricing Trends in Retail
00:37 - Introduction to Competitive Intelligence in Pricing Systems
11:22 - Competitive Intelligence in Retail
18:44 - Data Precision and Its Importance in Retail
21:16 - The Importance of Competitive Intelligence in Pricing Strategy
Foreign.
Speaker BWelcome to the latest edition of the Omnitalk Spotlight series, the series that highlights the people, the companies and the technologies that are shaping the future of retail.
Speaker BI'm one of your co hosts for today's interview, Chris Walton.
Speaker CAnd I'm Anne Mazenga.
Speaker BAnd today we are turning our attention once again to pricing because, hey, it's kind of in the news lately, right?
Speaker CAnd just a little bit, just a couple topics around.
Speaker CA little bit.
Speaker BYeah, a little bit.
Speaker BCan't get my mind off it, quite honestly.
Speaker BBut this time we are going to look at it from a new angle.
Speaker BWe are asking the question, what role does competitive intelligence play in designing an effective pricing system?
Speaker BSo we're going a little bit deeper.
Speaker BAnd to help us understand the answer to that question is with great pleasure that we introduce today's guest, Rajat Nigam, the Chief Product Officer at Clear Demand.
Speaker BRajat, welcome to omnitalk.
Speaker AThank you, Chris.
Speaker AThank you.
Speaker AAnd thank you for having me, folks.
Speaker AIt's a pleasure being here.
Speaker CYeah.
Speaker CWe have so much to cover with you today, Rajat, so we're going to dive right in.
Speaker CBefore we start, though, you're definitely an expert when it comes to competitive intelligence.
Speaker CSo if you don't mind, do you just want to give a quick background for our audience, for your background for our audience, Please, please.
Speaker AAbsolutely.
Speaker ALook, I would say that back in the day, I was hired for a secret project at Amazon.
Speaker BNot so secret anymore, is it, Rajat?
Speaker AThat is right.
Speaker ABut back in the day, Amazon or my team at Amazon was at the forefront of using machine learning and data to really reinvent how competitive intelligence was being used by a retailer like Amazon to lead the market.
Speaker AVery quickly, what I ended up doing was actually inventing the hardware, the networks, the software that was needed to go out to the Internet at scale and basically monitor anybody who was in the business of online selling to consumers and then monitoring what products they were selling at what prices, what promotions, what kind of shipping and convenience options, and really using that data to beat them at their own game.
Speaker AIt was massive amounts of data that were being collected, then cleansed, and then used by machine learning model to make automated decisions.
Speaker AThat was unprecedented at the time.
Speaker BWow.
Speaker BSo, so, and we've got, we've got the guy that created competitive pricing intelligence for Amazon via a secret project with us.
Speaker BSo I think, yes, I think Rajad is clearly, clearly an expert on this subject.
Speaker BAnd I think our listeners are going to get a lot from this conversation.
Speaker BAll right, so, so the next question that we logically have then is what in your mind separates good from bad?
Speaker BCompetitive intelligence.
Speaker BI mean, at this day and age, competitive intelligence seems pretty easy to come come by.
Speaker BCan't anybody just scrape everyone else's websites at this point?
Speaker BOr, or is, is there more to it than that?
Speaker BAm I oversimplifying it?
Speaker BWhat's your take?
Speaker AYeah, see, you're not wrong in saying that it's now easier than ever to build programs which can go to websites and collect some sort of data.
Speaker ABut getting data at scale is a very, very hard problem.
Speaker AFirst of all, I think the idea is doing it thoughtfully so that you're not taxing the resources of the people from where you're getting the information.
Speaker AMost people miss out that point.
Speaker AThe second thing is doing it in a way where you are not blocked by technologies like anti block solutions.
Speaker AAnd again, I'm not trying to say this nefariously, but again, but when you want to get data, you just want to make sure that it's predictability in getting the data.
Speaker AJust getting that bit right is a very hard engineering problem to ensure that you're trying to be like a human being going to a website or a mobile app and trying to get the information reliably.
Speaker AAnd again, without taxing a lot of resources of the competitor at scale, that becomes a very hard problem.
Speaker AThen finally, look, it's just one thing, getting data, making sure you're getting the right data and making sure that you can use the data that you're getting is a very different problem.
Speaker AI've worked with customers who worked with, you know, or were working with vendors, other vendors prior to us, where they would get heaps of data every day and they would sit on mounts of data just trying to scratch their hand, scratching their hand, trying to figure out what to do with it, how to make it useful and how to really use it for responding to the market to make things better for their consumers.
Speaker AAnd that's harder said than done.
Speaker BYeah, so talk to us a little bit more about that.
Speaker BLet's go deeper on that.
Speaker BSo what have you found that's useful to ameliorate some of those issues then?
Speaker ARight, so when you think about, from a retailer perspective, look, really what I want is that I want to increase the traffic in my store.
Speaker AI want to sell more units, I want to offer the best products to my customers.
Speaker AI want to offer the best prices, the best experience to my customers, really.
Speaker ASo that's the business problem I'm trying to solve.
Speaker AAnd I want to make sure that data or a data provider or a set of Technology can help me do that.
Speaker ANow, when you break that business problem into a couple of big pieces that you need to solve from a data perspective, is that a, all the comp intel data that you're getting is something that can be used with respect to the products that you are carrying.
Speaker ASo let me, let me break that down a little bit.
Speaker AFor example, let's say you carry Granny Smith Apple and you collecting data from your competitors who may be selling both Granny Smith Apple and honeycrisp Apple.
Speaker ANow you have to make sure that you match the right apples to be able to track the price correctly.
Speaker AThen by the way, retailers use different kind of pack sizes.
Speaker ASomebody may be selling apple pie, pound buy back, buy piece, and you may be selling it by piece.
Speaker ADoing the translation of that information that a price may be related to a different kind of unit of measure of sales to yours and then trying to normalize it to do a comparison is again a hard engineering problem that needs to be solved.
Speaker AYou get the data first hard problem, you need to then match the products correctly, whether it's the same products or similar products or related products correctly.
Speaker AThat's the second hard engineering problem.
Speaker AThe third hard engineering problem is normalizing these differences in terms of pack sizes and units of measure to make sure that you can actually truly compare the price.
Speaker ASo for example, again going back to my Apple example, let's say you had a one pound bag versus a half a pound bag.
Speaker AOne is priced at $5, the one pound bag and the half a pound bag is priced at two and a half dollars.
Speaker AYou need to be able to translate both the prices, add the half a pound bag that I am carrying to see whether I'm at the market below the market or leading the market in terms of prices.
Speaker ASo those are the three big things.
Speaker AAnd then again, once you have the comp and data, it's imperative that you use it in your pricing strategy to get the maximum return out of that.
Speaker BSo Rashad, I want to push a little bit on that too because one thing that I think Ann and I both learned in having these discussions for the past eight years around pricing is there's also the idea of, so there's the, the idea of, you know, getting the comp items right like you said, like, you know, apples to apples, for lack of a better way to put it, and then pack sizes and whatnot.
Speaker BBut the items themselves don't exist in isolation because oftentimes they're, they're a part of a larger category.
Speaker BPrice positioning too.
Speaker BSo is that also part of of what you need from a competitive intelligence standpoint to understand how the product item at the item level fits into the category pricing strategy as well.
Speaker ABecause.
Speaker BBecause the actions you take could lead you in the wrong direction if you're not thinking about it in that way.
Speaker AAbsolutely, absolutely.
Speaker AI think you're right.
Speaker AIt's not just looking at the apple, but the category as a whole.
Speaker ATrying to understand the category architecture of not only your category architecture, but how does your category architecture works with respect to your competitors category architecture.
Speaker AWhat kind of brands, what kind of pack sizes are they carrying?
Speaker AWhat's the price ranges of the products that they typically carry?
Speaker AWhat kind of an audience are they appealing to?
Speaker AWhat locations are they operating?
Speaker AAnd all of this sort of becomes imperative and becomes a part of, you know, part of the execution metrics, if I may say so as we work for the customer.
Speaker CSo Rajat, I.
Speaker CHow then should retailers be thinking about competitive intelligence?
Speaker CBecause I think on one hand you and Chris were just talking about making sure that your prices meet or match, you know, or at least are aware of what your competitors prices are.
Speaker CBut as we get into, you know, things like promotion strategies, as the economy is going up and down, we're getting into other things that retailers might need to, to use this competitive intelligence for.
Speaker CWhat, what else are retailers that you are working with?
Speaker CWhat else are they using this for?
Speaker CAnd like how do they kind of use that to justify maybe the expense at first, like the first like capital that they'll put into a competitive intelligence platform?
Speaker AWhen you look at competitive intelligence data, you are getting all the product information, the category information, the pricing information, promotions, convenience, shipping options, all of that, you know, are the third party sellers selling the product on the platform.
Speaker AYou know, what is the, you know, you get a lot of rich data and, and you could do multiple things with it.
Speaker AYou know, pricing is the first one where most customers start.
Speaker AI think that's where you can add immediate value.
Speaker APromotions is typically what follows next.
Speaker AOkay.
Speaker AOr in parallel, I have seen people doing assortment gap analysis and management.
Speaker ASo again, you want to make sure that you're carrying the best products that your consumers want to see in the store.
Speaker ASo you try to figure out what is it that I'm not carrying that the other guy is carrying that may drive more traffic to my store and at the end of the day delight my customers.
Speaker ASo assortment gap management is again our engineering problem.
Speaker AWe're trying to figure out and answer the question what should I buy next?
Speaker AAnd again, by all means, you understand building supplier relationships because of to Manage those assortment gaps, then onboarding the product, then productionizing them, making sure that you have a supply chain to get them into store.
Speaker AAll it's a big decision to figure out what do you want to sell next in your store.
Speaker AAnd assortment gap analysis is the first step that basically helps in that decision making.
Speaker AAnd then, you know, the other thing that I have seen customers, you know, my, my, the retailers that we work with use the data for is of course, a regional planning of assortment.
Speaker AYou know, depending upon, you know, again, the market and again the audience that you're targeting, you may want to have a distribution of products across the country and across the regions that you operate very differently depending upon which category, what product you're talking about.
Speaker ASo that's another thing that retailers do.
Speaker CAlso, you know, as we look into ahead, into the coming months where there's a lot of uncertainty in the market, can retailers still do this the way that they were doing it without a competitive intelligence platform?
Speaker CI mean, you've named like seven different engineering problems that you're trying to solve.
Speaker CIt just seems like a lot for someone to manually be doing or to be, you know, doing through a disjointed process.
Speaker CWhere, where do you, how do you think about that?
Speaker AThis is a signal that you can no longer ignore looking at competitive intelligence and looking at your market position.
Speaker AAnd you have to do it more often, more often now than in the past.
Speaker AI mean, gone are the days where you could send, you know, a mystery shopper to a competitive store once in a month or once in three months to benchmark your prices, that those days are over.
Speaker ASo, so that's the first thing.
Speaker AThe second thing is I think it's always good to start small with a good partner that you can work with.
Speaker AYou know, have simple objectives like, you know, focus on a few categories that you want to lead with, a few KVI items or key value items that you want to focus on to improve the, the price, value image of those items compared to your competitors and then scale up.
Speaker ASo start with pricing, then promotions, then assortment, gap management, and then look at other problems.
Speaker AThe data that you have is still going to be gold.
Speaker AThe data like, even if you start from pricing, you're not going to miss out on anything.
Speaker AYou'll have the historical data which you can back and look at and then use for other things later on.
Speaker ADid I, did I answer your questions?
Speaker BYeah, I think you did.
Speaker BI mean, I think it's actually a big nugget for, for, for us, I think, you know, and for our listeners too.
Speaker BBecause what, what you're saying, what you're saying essentially Rajat, which I don't think was clear to me before this conversation, is that, you know, competitive intelligence, while a necessary condition for good pricing, intelligence is also a necessary condition for a lot of other things like the quality of your assortment, the speed and convenience at which you deliver that assortment to people, particularly E commerce space and other aspects of operations too, I would imagine.
Speaker BSo, so I think yeah, 100.
Speaker B100% you did.
Speaker BBut my question though, actually my next question though is, is again kind of putting you on the spot.
Speaker BNo, I get how this makes sense for you know, for grocery particularly.
Speaker BLike I think it, you know, it makes sense as you're describing, you can get a sense of all those things in terms of how you're stacking up competitively.
Speaker BBut is this same rationale philosophy applicable to all areas of retail or are there areas that it doesn't work as well for?
Speaker BI'm curious what you think about that.
Speaker ALook, I would say that for all high velocity retail segments, whether it's grocery bats convenience, I mean segments where consumers often make purchases, you know, on a recurring basis in short amounts of time is definitely where, you know, value perception of value by consumer is a very, very big thing.
Speaker ANow when you move to the, to the, to to other segments like high end fashion for example, I think the need sort of disappears because you're trying to, you have a very different objective there.
Speaker AWhether it's expensive watches or are very high end shoes, I think the dynamics change a little bit.
Speaker AI think the user definition of value and per is owning a branded piece there versus you know, when you come to high velocity, high velocity segments where they basically they want us like nobody wants to buy cheap products, everybody wants to buy high quality products, but they want to make sure that it's the value that they get for every dollar they spend is maximized.
Speaker ASo I think it's just very two different objectives as you go through different retail segments.
Speaker ABut then again, I think I firmly believe that for all high velocity segments, the pricing as one of the biggest pillars for driving value to your customers, I think is the truth.
Speaker BAnd that notwithstanding too, I imagine even in the high luxury fashion area, having competitive data on how well you're shipping or what your times to ship are relative to others is still valuable.
Speaker BBut I'm curious, I want to ask you even more.
Speaker BHow does price elasticity come into that too?
Speaker BIs it just high unit movement or is it also where there's, you know, a lot of price elasticity where the competitive pricing is also important.
Speaker BLike I think categories like electronics, they might not move as quickly, but being, you know, lockstep with the market in terms of where your price and your assortment gaps are, is pretty critical.
Speaker BWhat do you think about that?
Speaker AThat's again a great point.
Speaker ALook, I mean you could have the best assortment, you could have the best price, but if you have empty shelves and poor consumer experience, that's no good.
Speaker AYou know, I think elasticities help you with better price planning.
Speaker AI think they also help you with better demand planning to make sure that, you know, your consumers always have a good experience, that they have the products always available, that they want to shop at their convenience.
Speaker AAnd you know, basically you're meeting the consumer and in their buying journey every time.
Speaker ASo elasticities do definitely play a good role there.
Speaker AAnd then, you know, computing elasticities, again I would say is a, is a hard engineering problem to make sure that you're doing it right.
Speaker ASee the thing is that, and most people don't realize it, that elasticity change.
Speaker ALike imagine this, you're looking at two years worth of data today and it signifies a certain elasticity and then you make a price change today with response to that and then, you know, it may take up to next couple of weeks before the true effect of that new price would, would basically start reflecting in the new elasticities.
Speaker ASo it's a moving target that you need to know how to leverage to, to, to maximize, you know, taking the benefit from that.
Speaker CSo, so Rajat, if, if people are listening and they're, they're all in, they're thinking, yes, we need to make a change in this direction towards the types of things we've been talking about.
Speaker CHow should they go about doing it?
Speaker CWhat considerations should they be making?
Speaker CWhat assessments should they be making before they take the leap here?
Speaker AA good partner can of course share best practices can basically, you know, reinforce or revalidate, you know, what you might be thinking as your go to market strategy for both pricing and comp.
Speaker AAnd so I think I would start there, starting with a good partner.
Speaker ABut, but you know, once you have a good partner, I would again say, you know, focusing on a good product mix, especially KVIs, fast moving items that, you know, basically drive the maximum amount of your revenue or margins or, or categories where you actually want to turn around things like you're tired of, you know, you know, losing money and you say look, I want to change things here to see some movement and turn this around in terms of profitability or margin.
Speaker AThose may be two Good candidates to, you know, two different kinds of candidates to start with and move forward.
Speaker BHow do you, how do you help the retailers on the data side of this?
Speaker BLike the one thing we always hear and it's always such an amorphous, amorphous topic.
Speaker BNow credit to the retailers that talk about it first of all, because a lot of them don't, which is, you know, the idea of data.
Speaker BSo like how do you get the data around this and what aspects of data do the retailers need to consider to implement this idea correctly?
Speaker AChris I think the most important thing here is when you think of data is precision of data.
Speaker AThe data quality is one of the key metrics I would say that customers should insist on, irrespective of who they're working with as one of the key metrics.
Speaker AThen I think it's the completeness of the data and the data coverage.
Speaker ANow for example, you may be a regional retailer operating in three or four different states.
Speaker AIn every location you are operating, you may have different kind of competitors around you in a 1 mile radius, a 2 mile radius or a 3 mile radius that you want to monitor.
Speaker ANow it's not just sufficient for your partner to deliver data for one region.
Speaker AYou have to make sure that all competitors, all regions are covered, covering all the products across all the categories to begin with.
Speaker AThen that data is basically used by a price optimization engine like ours to basically help you then come to the best prices possible.
Speaker ABut look, at the end of the day, it's garbage in, garbage out.
Speaker AYou have to collect all the prizes, the right prices, you have to do good product matching, be accurate at it, and then basically also do the right normalization of the competitor prizes before all of this gets fed as input into, into the good optimization work that a pricing engine can do and then make, you know, then output for you the best prices that you, that you need where you could truly see the impact.
Speaker BYeah, that's a really, that's a really good way to frame it up too.
Speaker BI think for our listeners too, they're like, okay, because that's always the question we get right and is like where do I start?
Speaker BAnd like, yep.
Speaker BAnd to your point, you can start in any category you want to with this.
Speaker BBut you've got to think about it from the framework of where do I have good data quality, where's the data complete?
Speaker BAnd then where do I have the coverage?
Speaker BAnd you know, and you've got to do kind of a gap analysis before you start this to understand where you're going to have the most effectiveness when you try to implement this solution.
Speaker BSo I think, I think it's really smart.
Speaker BAnne?
Speaker CYeah, I mean I, I think I'd love to close Rajat with the, like your kind of predictions and thinking about where, where retailers who are investing in, you know, competitive intelligence platforms and those who are not, where, where does the future lie for them, especially in regards to pricing mean, what's the end result here?
Speaker AYeah, I firmly believe that competitive intelligence is one of the most important input into your pricing strategy right now.
Speaker AWhether or not you choose to match prices with your competitors or whether or not you don't want to be the prize leader in the market is a very different question.
Speaker ABut irrespective of what kind of high, low strategy or everyday low prices strategy that you have as a retailer, the reality is that you have to make sure that the perception of value that your customer gets by shopping in your store, which is a combination of price, experience, convenience, quality of products, that has to raise the bar with respect to your competition.
Speaker AAnd you cannot ignore Comp intel, which sort of covers a lot of those bases as input data anymore.
Speaker ASo my recommendation to all our retailer fans is that if you're operating in a high velocity category, definitely look at Comp intel data as a key input strategy.
Speaker CWell, and, and Raja, just one more thing.
Speaker CLike I think you have to be with all of the, the constant changes to pricing right now and like you said earlier, you know, making sure that you have the flexibility as a retailer to make sure that you adhere to certain pricing rules that you've set up for yourself, like in produce.
Speaker CLike we're always going to be, you know, we're going to always going to try to hit the price in produce so that we can get people in and maybe we can adjust margins on some of the other things that we have in our store.
Speaker CBut like, how important is it that you have that agility in pricing as we head into the future?
Speaker AYeah.
Speaker ASo look, I will tell you that there are categories that we operate in where retailers are changing price at least four to six times in a day.
Speaker CWow.
Speaker BWow.
Speaker ACross key value items.
Speaker CSure.
Speaker AThis is not during the holidays.
Speaker AAnd during the holidays the war can become so intense where prices may be changing every five to 15 minutes across.
Speaker ASo it's imperative that you are able to respond to them.
Speaker AAgain, not every category is the same, but we are at a point where we see that daily.
Speaker AMost of our customers, especially in the grocery space, want the data daily to be able to see the changes in the market and be able to respond to them.
Speaker AAnd especially with as technology is evolving around us, for example, the electronic shelf labels.
Speaker CSo look, I was going to say.
Speaker AYeah, that's going to accelerate the adoption of price optimization and competitive intelligence even more.
Speaker ANow, initially you had the friction of somebody going in, sticking to labels, all of that, but now, hey, you know what, you know, you got a husband, shell fetch, you're gonna do great in terms of responding to the, the market, you know, without even thinking about it.
Speaker AIt's all automated.
Speaker BYeah, yeah, right.
Speaker BYeah, yeah, no, yeah, that, that's what it tells me, too, is like if people are trying to change prices that much online, whether you know what, you know, they're going to want to start doing that in store and they're going to look to, you know, companies like the Vision Group, like we cover on our show all the time, and try to figure out how to do that so.
Speaker BWell.
Speaker BThank you, Rajat.
Speaker BThat was really, really interesting.
Speaker BA lot of nuggets from the conversation.
Speaker BI think I said the nugget, the word nugget, like three or four times, maybe, or more, who knows?
Speaker BBecause I never thought about, I never thought about pricing intelligence only being as good as the competitive data that feeds the algorithms.
Speaker BI mean, it's intuitive, but it's something that I know I personally had never thought of.
Speaker BSo if people want to get in touch with you or anyone else at Clear Demand, what's the best way for them to do that?
Speaker APlease definitely reach out to us on our website.
Speaker AOur website is www.cleardemand.com.
Speaker Aplease request for a demo.
Speaker AThere's a easy form to fill and we'll promptly get back to you.
Speaker CThat sounds amazing.
Speaker CThat wraps us up.
Speaker CThank you so much, Rajat Nigam, for sitting down with us today.
Speaker CAnd to everyone out there who is listening in, and as always, on behalf of all of us here at Omnitalk, be careful out there.