AI Will Never Again Be This Cheap

Bren: Welcome back to This Is Definitely Not a Data Podcast. I'm Brendan and as always I'll be joined by my good friend Leno, who sadly still hasn't been recognised in the street. Today we start somewhere small with the actual cost of running AI, and whether the cheap tokens we're all enjoying right now are just the free trial before the massive price hikes. We're also going to get into a much bigger question for what happens to the millions of people in offshore centres if agentic AI actually comes for their jobs? Is this a real risk? or are we just catastrophizing? Hello and here we are again. If you don't know who it is, my name is Brendan. I've got a very good friend of mine, Leno, with me. Hey Leno.

Lenno: Hey Brandon, good day! How are you?

Bren: How about how about this? You know what? I mean it's been it's been a while. It's been a while, but we are we are back. I'd like to say with the next episode. I don't know whether anybody's young enough to remember Dr. Dre and the the next episode song, but here we are. We are kind of whistling through season one of this is not a day to podcast. but Have a listen and let's get into it. I think the feedback we've had has been good so far. So we are you know, we we are one step away from challenging Joe Rogan on being a worldwide podcast phenomenon, which is very, very exciting. if you believe that, you will probably believe anything. So we shall probably kind of gloss over that. But I hear y have you been stopped in the street yet in ⁓ in your hometown, Leno, asking for autographs or anything?

Lenno: Bye bye. I've been stopped by the police and stopped by packaging deliveries to put an autograph somewhere. But apart from that, I think stardom is not yet there. Which is totally fine. As long as this kicks off some level of motivation for some people around, I'm okay. You know, I'm just...

Bren: Yeah. How about that? It's it's only a matter of time. Good.

Lenno: enjoying the conversations that we can have and ⁓ there's a microphone here, okay, that's recording our conversation, apart from that, you know, it's just having fun.

Bren: It's it's it's actually very, very true. That the the conversations that we have and and and hopefully will resonate to to those of you listening is th these are conversations that we've had behind closed doors, that we've had in certain kind of conferences and and actually we just think that that that they would be valuable because if we are and I'm not saying that we've got everything right, but if we are having these conversations, I'd I'd I'm willing to guess that others are having it too, so

Lenno: and

Bren: As long as you're happy for me to to put a microphone in front of you, Leno, I am happy to keep keep doing this.

Lenno: I'm totally okay with that and I'm very glad that you've just mentioned that I'm not expecting to have everything right now. I don't have all the answers. I have a whole lot of uncertainties. We're still talking about data, by the way. ⁓ This is not a psychology thing that we're having over here. Yeah, exactly.

Bren: What? You don't have all the answers. Yeah. That's a different podcast, isn't it? That's l let me begin with Leno. but no, I I think it's it's it is important and and the conversations we have, which is why we we we find them we find them so valuable and which is why you're they're coming through your ears. But today I wanted to I guess I really wanted to put your brain, Leno, on kind of the cost of AI. Because okay, I mean we've probably turned off Half our audience by talking about the cost of AI, but let's let's go on the presumption that the other million people are still listening. ⁓ I had a conversation. So I'm I'm just gonna explain a bit of a situation and then tell me what what what you think. Had a had a conversation last week where talking about use cases for AI. And often if all you've got is a solution, trying to find a problem is hard, and that's often what's happened with with AI. I was talking to this individual. And they were looking at where best to implement AI. And it turns out that they've got a very good use case that is quite a critical process for the business. So there was a a a a piece of work going on where they were looking to how best to implement it in that process. So far so good. The question I I posed was what happened Well h I I guess, how much have you budgeted and and and what happens if costs increase. And the reaction surprised me somewhat, which is why I want to sense check it with with you, is that a bit like mortgage rates were in the UK over the s over the sort of COVID, they they were as low as they've ever been. They were never going to get any lower. So when people were surprised that they went up, I I didn't understand that. And I I have a same feeling with with AI is that AI is not going to get any cheaper. So in a year's time that process that could cost a hundred dollars to run might cost five X that, or whatever anthropic and the teams decide. Is is that a real concern, or am I just sort of overplaying and and and the encounter in me coming out a bit too a bit too strongly?

Lenno: Mm-hmm. Although I've not encountered it specifically yet, ⁓ just like you're saying, the cost to run models right now is relatively low. If we indeed were to reference other situations like you've mentioned mortgage rates ⁓ or put this into energy, instance, energy or ⁓

Bren: Mm.

Lenno: SAS solutions in general. then I think you might be on to something because all of these prices fluctuate, will go up and there may be situations where it goes down.

Bren: Mm.

Lenno: that in principle I think in the you're having a finance background right? okay

Bren: Yes. Yeah. I I mean I try and keep it quiet 'cause it'll send people to sleep, but yes, I I'm an account I'm an accountant by trade.

Lenno: ⁓ I think what you will be seeing on many many years is that effectively the price levels go up and therefore you need to constantly ⁓ transform yourself constantly offset and do things differently, look for measures to reduce it again. This is where shared services at some point kicked off and therefore positioning jobs to countries where the rates, the salary rates are lower. This is where this and I would also argue around being able to sustain your company's resources domestically and the inability to do so, a whole lot of offshoring and hyperscalers, SaaS services, et cetera, are flourishing. Now the flip side of that is that we are making ourselves dependent on suppliers, support partners as well. To your point, implementing something in my core process makes me highly dependent on the continuation of service delivery by someone. Now, what we've seen in the early launches, of ⁓ great technology companies. You you get flyers like, you get six months for free and then you will be paying like a flat rate of whatever that may be. So the early adoption is high and then all of sudden the payment cycle kicks in. And then year by year the cost goes up.

Bren: Mm.

Lenno: ⁓ is that price elasticity elasticity so please help me there difficult work for me exactly yeah what can people still

Bren: Yeah. Price elasticity. There you go. Okay. But just don't don't make me do this in in in Dutch, because I'm very conscious that you your your English is excellent. ⁓ but just I'll help happily help you out. Just don't make me do this in return the other way around.

Lenno: No, for sure. ⁓ No, ⁓ I won't put you on the spot for that long. And is there a breaking point somewhere where the price becomes a dissatisfier? And that may very well be the case. think we've seen situations where that's happening and people then will start to step away.

Bren: I've I've seen from a personal perspective, I mean I rightly or wrongly I pay for the pro for Chat GPT and for Claude. Claude especially, you what you used to be able to do before hitting the usage limits was much more. And I mean I I'd say you know, not that I've ever had crack, but you know, you want to get somebody hooked and then you kind of slowly reduce what you get and then you increase the price.

Lenno: Mm-hmm. Yeah.

Bren: it you know, obviously neither confirming nor denying, anything like that. But y you can see how I mean Claw just went on anthropic was mega. I mean they just got everybody using it hooked on it. It it it took the world by storm. And quite recurringly now I'm hitting usage limits or session limits on what can be done. So it it made me wonder that, you know, my my cost hasn't changed in the sense of I'm paying the twenty pounds a month.

Lenno: Thank Yeah.

Bren: But then you can pay extra for the for the additional session time that you want. But you the the number of queries and the I guess the tokens you're using are becoming less. Yeah. So so when when you're thinking about and that kind of casts me back to the the the situation I was having ⁓ the conversation with, in in your role, is cost a a key consideration when you're looking at how best to leverage AI?

Lenno: ability becomes less. Cost is a big thing in everything that I'm Especially in the company where I'm working at this moment, ⁓ cost is a big, big element. And we need to get the cost down as much as possible. Nevertheless, achieve the maximum with that every euro or dollar that we're spending. That's probably the case in any company, by the way. Everybody needs to spend very smart. At this moment in time, the token cost may be low or limited. to your point indeed we're now hitting we may be hitting technical boundaries where there's so much adoption that all of a sudden the what is it the latency or the usage ⁓ is going to be tuned back still allow a rather large ⁓ bandwidth of people using to still use it But effectively, what happens is that your cost goes up because your ability goes down.

Bren: Yes.

Lenno: ⁓ I liked your comparison with the drug dealers by the way. Get everybody hooked and then slowly increase the prices or tune down the usage. And we've seen that in retail ⁓ for many many years as well. The price if

Bren: Yeah.

Lenno: Either the prices are being increased or the pack size is being reduced.

Bren: Mm-hmm. Yes.

Lenno: And that's actually very, very delicate, maybe smart or just wrong way to cannibalize on your brands.

Bren: Yes.

Lenno: So this is where in retail nowadays, for instance, and especially over the course of the last couple of years, where energy prices went up through the roof, labor prices went up through the roof, and therefore there's a high degree of cost pressure. A whole lot of consumers switched from A brands into white label. And thinking about that out loud, it might be a point where the A brand AI suppliers now will need to start competing with a form of an AI white label, maybe open source or something like that. Because of the elasticity.

Bren: Mm.

Lenno: And absorption ability of people is no longer there. The affordability becomes too high. Now, coming back to your example, where you've mentioned ⁓ your colleague that implemented AI in the core business function. I would argue that any AI type of initiative or any initiative, you also need to take a look at how quote unquote easy can you transition from one support partner to another. And maybe that's a sustainability part for a company as well.

Bren: Yeah. Actually y y you've got my grey matter going 'cause I think you I think you might be right. The I guess the the w what what you've made me think of is the the the concept of the one way, two way door decision. And I mean for for for a bit of background for for people listening, a two way door decision is something that can be easily reversed. So the the the principle is that you should make it quickly because you can you can reverse it. So little example, you walk out the door, you're gonna turn left, you realise ten meters in, you should have gone right, you've only got ten metres to come back before you've course corrected.

Lenno: Okay.

Bren: Two-way door decisions, you see them ⁓ across businesses all the time. A one-way door decision is something that can't be easily reversed. Now it used to be the old implementing Oracle because you're in for ten years and tens of millions of dollars. Now, I would argue also that your choosing your cloud provider is no longer a one-way door decision because all y you leaving Microsoft to go to G CP or AWS painful, but you could do it. Now I wonder whether

Lenno: Yes.

Bren: We're getting into some serious one-way door decisions where if you implement let's say for argument's sake, you implement Anthropic into your into your processes because it's the best that you've got at the moment. Uber apparently burnt through their entire AI budget in four months because they went back crazy over token usage. What are you gonna what are you gonna do then? Because that that is a one-way door to see you are you are hooked in and you are stuck.

Lenno: Well, imagine that indeed you've implemented a hyper automation into your core process. And by that you've, you've automated a workforce of 50 people. And this is hypothetically speaking. And then all of a sudden you realize that you've burned through your budgets in half a year. You still have six months of.

Bren: Yes. Yeah. Yeah.

Lenno: business opening to support. Are you going to lay off your workforce of 50 digital employees? Substituting by what? Because the people have left have probably repurposed somewhere else because you've been able to digitize your entire company.

Bren: Well. Yep. But but but what what about that the whole business case was probably predicated on we're gonna let's say we're not gonna rehire or we're gonna you natural attrition or even we're gonna let people go. Implement it into the process. The r the business case was signed off on it being inadverted commerce cheaper. Well, in a year's time it probably won't be cheaper than having those people do it manually. And it might even be considerably more expensive. But you're now doing l Well, you're gonna have to deal with more, with less people, with more cost?

Lenno: Yeah, that's a great one. Makes me think of how business cases indeed nowadays are being put together. You're basing your return on investment indeed on the actual prices with a form of an up mark, a markup. Do we need to take into consideration a calculation mechanism where indeed in three or five years time AI cost is going to X5 or X10 instead of the typical 10 or 15 % on an animal basis?

Bren: Yeah it could yeah, great point. Because i historically I guess we've all gone, ⁓ the license, you know, it's probably unless you happen to be a SaaS provider where it seems to go up thirty percent year on year, you're like okay, yeah, ten percent, fifty percent makes sense, you can kind of budget it. I I think you're absolutely right. That those those days are gone, certainly from an AI perspective. But also, if you're planning out more than two years, I I think it's all a bit fictional. Because I mean, if think about how far we've come in the last twelve months, imagine what the next twelve to twenty-four could look like as well. From a a sustainability perspective, that that is a a big question. But you you you touched on something I'd or or mentioned something I'd I'd like to just kind of quickly mull over. When it comes to replacing gr critical processes with AI, there's been a big tendency in the past to offshore things.

Lenno: you

Bren: Yeah, the sort of the the offshore locations, the Manilas, the Mumbais, the you know, the the the the the big setups. Well I would imagine that agentic AI is prime for following step-by-step processes, which is where offshore centres of and and locations have historically provided value to onshore firms. If if AI does what we think it could do, it

Lenno: well.

Bren: Are we saying that those people in those locations are less needed? Ergo, you don't need a team of or an office of 200 people doing manual process work because the the agent can do it, and you might need one or two per I don't know, fifty, seventy agents, I don't know what it might be. But if that is the case and you follow it through, that's the displacement of millions of people.

Lenno: Okay.

Bren: 'Cause if you live in Mumbai you can't easily remove or retrain to Chennai and and so that that to me is a really big worry that I'm not sure we're we'll we'll we're we're talking about.

Lenno: Wow. Wow. Now you're opening up a can of worms. Just thinking about it, the hyper automation has the tendency of hyper automating repetitive ⁓ and to some degree generative processes, which ⁓

Bren: Mm.

Lenno: thinking out loud in these where shared service centers have been ⁓ extremely successful in that particular space.

Bren: Yeah.

Lenno: And these were the processes that effectively all of the back office functions can be brought into shared service space. We use effectively across the company, the entirety of the company. These are similar type of patterns that can be executed at a lower cost in comparison to US, Western Europe. And by that, to your point, there's a high degree of employment. in geographies where at this moment time labor cost is still relatively low in comparison to other countries. If we then indeed put in AI hyper automation into the mix and have these processes being executed through generative and or automated processes with virtual workforces.

Bren: Yes.

Lenno: would that indeed cause the likelihoods may very well be there that it causes massive unemployment in these geographies?

Bren: Mm.

Lenno: And then the question becomes, how do we deal with that globally? How do you sustain that? How are we repurposing these geographies? there a responsibility to also think about that and take that into account while automating your businesses?

Bren: Yeah. My my my initial reaction is th we we we need to. But I I've not met many people slash any, may maybe a couple, who have had a similar thought process and come to the same conclusion in independently. So you think well i implementing AI is obviously a really hot topic and there are definitely benefits around it and I'd like us in another episode to talk about sort of ROI on AI because I'm really I've got mixed feelings around around that. But if we are going gung ho and and and putting in AI That was embarrassing. So hopefully ⁓ that didn't come through. if if we are trying to do what we want to do and we're gonna go all in on AI, there are going to be casualties. And if I don't I don't really ⁓ So no, I don't really know because the butt the part of me says, am I catastrophizing? It's like watching a like a disaster movie, be like, well, if it all goes wrong, it's all gonna go really wrong. But i is is this a is this a genuine piece? Because i if it is, we need to consider it more. Should should we hold back on what we want to automate? I don't think so. But should we be more aware of the sustainability of using AI? Very much so.

Lenno: I think the economic cycles of the way that we're ⁓ wired, the way that companies operate, we need to evolve. And therefore, I think AI is part of what we are doing nowadays. And it's going to help us in that economic evolvement.

Bren: Yeah.

Lenno: evolution, the sustainability aspects of AI in general. I have a lot of question marks around it. Coming back to what you've mentioned at the start also, you're now tuned back in your ability. Is that because the user adoption became bigger and therefore

Bren: Mm.

Lenno: the bandwidth doesn't allow that to ⁓ function to the level that you used to. Is that because of a conscious choice of the AI provider tuning back your ability and by that gaining the same income ⁓ per person with providing less services. Energy consumption in general of AI across the board goes off the charts. On a daily basis, you can effectively run a country with the energy consumption. And I'm not talking about a small country or country where electricity is yet to be evolved.

Bren: Yeah.

Lenno: This is like established areas where energy consumption is quite high already. energy, I believe is challenge. ⁓ Cooling, which genuinely is being done by water, is another. And these two elements are triggering scarcity and are triggering sustainability questions in my mind, to say the least. And then indeed, what we just stumbled upon in our conversation is how do we deal with potential massive unemployment? ⁓ rounds of layoffs in companies we've seen that for many many years which is very difficult for the people who are affected I do believe, and I'm saying that very very consciously and cautiously, that the majority of people find a new path after that. maybe also here with implementing AI at large scale, the adoption of AI in large scale, we may be looking at layoffs on an even bigger scale. And then the question in my mind that comes up is... Is that indeed a sustainable and an absorbable pattern? I don't know. This may be so significant that we can't.

Bren: my fear is that it it's the latter. And I guess a little bit of background. I've I've fallen foul, you know, I've been impacted by layoffs twice and been fortunate enough, you know, living in South West London, being an accountant by trade, you know, been fortunate enough to find alternative opportunities. You know, it's hard, it's painful, but you you can do it. If you are reliant on working in a in a in a an established offshore centre that's you know in Manila or Chennai or Mumbai and and and that's that's your family's livelihood. And that gets taken away by something completely outside of your control. Not only have you got to deal with that, but you've then got to work out what what one does next. And the sustainability part is this is this is millions of people that could be displaced. But the question Ha have you if you mentioned this at your

Lenno: Yeah.

Bren: at your exec level. I w what do you think the reaction would be? Do you do you think the reaction would be we need to consider it or it's not my problem or it's not likely to happen or or are we what do you think the the kind of the f the feedback would be?

Lenno: Whoa, it's an interesting one. I've not thought this through, let alone post this question in the executive forum.

Bren: Yeah.

Lenno: The last time, and this is a couple of weeks ago, when we were talking about adopting AI and implementing AI, exploring AI for our company, I did post a question around what is the sustainability ⁓ pattern that we want to adopt also. We have a very high degree of focus onto ESG.

Bren: Mm.

Lenno: And by that focus on what is our footprint across everything that we do. I said, OK, so when we're talking about AI, we also need to take into account how we are thinking about offsetting energy, water consumption of the models that we're using. How are we going to do that? ⁓ yeah. OK. So the response was, ⁓ yeah, that's a good point.

Bren: Yeah.

Lenno: It did blow a little bit back on the excitement that was in the room on exploring and adopting AI because effectively it's the new shiny thing that we want to do. So why are we now making it difficult on... And that wasn't said, not at all. It's a bit of a mood killer.

Bren: Interesting. Yeah.

Lenno: Also, ⁓ yeah, we also need to take into account sustainability aspects of it. And maybe it's not only energy and water, maybe it's also the social element in terms of what will this do to workforces in different geographies and domestic.

Bren: Hi historically ⁓ yeah, yeah. No, I was gonna say I think historically companies and ⁓ I've definitely fallen fell out of this in the past, is we've with ESG we focused a lot on the E and not a lot on the S and the G. And now i but y but your your point is absolutely right. W it feels a bit like being a party pooper by saying, ⁓ with all the shiny new object stuff that you were ⁓ imagine all the cool stuff we could do.

Lenno: I don't know mate. Yeah. Yeah.

Bren: But what about the real human impact? I'm like, well pff but we do need to consider that because it is I think and this is by virtue of getting to the the senior levels of a business, you have a responsibility to look at and try and preempt some of the big questions that you need to address. So I I know that in previous lives, when I've said who's going to be the monzo or the Netflix of our business or our industry, and how we're going to deal with that, the reaction is, ⁓ it's not going to affect us because we're a relationship-based business. I was like,

Lenno: Yeah. Mm-hmm.

Bren: Okay, well you're gonna be dead in the water then, aren't you? I it it'll happen and you'll you'll you'll complain about it. So I think Well then yeah, e exactly. You'd be like, why didn't we see this coming? It's like, well we did, you just chose to ignore it. But the ⁓ now, as as I guess as we get to senior levels and we have these conversations, it's like, well, okay, what should we do?

Lenno: Yeah. We will have our codec moments.

Bren: Or even if the answer is we don't know what to do, let's park the question, we can't ignore the question. I guess that's really where I've I've kind of landed in 'cause w e everything you said ⁓ makes a lot of sense and and as I say that the mind is going a bit a bit like the clappers at the moment, 'cause you're like, well, if this happens, if this happens, if this happens, if this happens, then that happens, you're like, ⁓ my word, we're in trouble.

Lenno: Mm-hmm. Maybe we can ask an AI to simulate this?

Bren: That true. True. Or just watch Terminator two and then Skynet will But no, yeah, y I mean it's it's a very it's a very sensible point. I I think it'd be interesting for anybody listening, ⁓ just to kind of send us your thoughts. I think that the the bit and and I feel like we're we're about to end on a slightly somber note of sort of mass unemployment, but it's not it wasn't meant to be that way. It was the the whole concept of

Lenno: ⁓ yeah, yeah, yeah, yeah, yeah.

Bren: This is not a data podcast, it's around asking the questions that we think should be asked. And I mean, Leno disappointed me when you said you don't have all the answers, which was you definitely didn't say that right at the beginning. But I I think it's I think it's a really a really, really pertinent question and certainly something that we're gonna have to deal with as as technology goes ahead. If it but before we

Lenno: Yeah, sorry. You need to find someone else who does. Yeah.

Bren: Before we close out today, have you have you have you got any closing thoughts? Any anything you wanted to kind of end on?

Lenno: I think it's really interesting for me to recognize where we started tangibly around cost increases of the providers.

Bren: Hm.

Lenno: And then evolving towards what would this also socially mean in terms of responsibility and taking responsibility, taking accountability in everything that we do. Feeling a piece of care, I think needs to be in the core of everything that we do. We need to care enough. to at least articulate the questions and carry enough to then think this through.

Bren: Mm.

Lenno: And that again is not a data topic. It concerns people.

Bren: Yeah. And and hence, you know, the the marketing department were working overtime when they came up with the title of the of the podcast. But you're absolutely right. my belief in the world of AI, the people aspect is be gonna going to become and if not is, the most important factor that we need to consider and and I and I could not have put it better myself, Leno, but as as we bring another episode to a close, please do if you're listening, like and subscribe, please do leave us a review. Leave us a five star review. That's what that's what we're really after. Just tell us how great we are. ⁓ it will do something with the algorithm and and AI will be really nice to us and all that sort of stuff. But

Lenno: Yeah.

Bren: little flattery will hopefully get us everywhere, but is again, I cannot believe how quickly time has gone, Leno. That has been an absolute pleasure. but yeah, thank thank you. That was wicked.

Lenno: Again, thank you very much for hosting, guiding the conversation. And I'm so much looking forward to the next time.

Bren: And with that, we'll see you all in the next episode. Bye for now.