Fernanda Dobal: proving ROI with AI chat and assistants
In this conversation, Steve Budd speaks with Fernanda Dobal, the newly appointed product director of AI and chat at Cleo, a fintech company. Fernanda shares her experience in using AI to develop AI assistants for sleep training and financial coaching. They also explore the importance of user trust and the role of humans in the loop. The conversation emphasises the value of AI in improving efficiency, personalization, and accessibility.
AI Pathfinder Private Equity AI Strategy Meet-ups:
https://www.aipathfinder.co/private-equity-ai-events
Have a question or would like to suggest improvements, please contact steve@aipathfinder.co
Hello everyone and welcome to the AI Pathfinder for Private Equity podcast. I'm on a mission to lead private equity from AI bewilderment to AI empowerment, and I'm doing that by building a dedicated network that will piece by piece create a playbook for successful AI adoption across PE operations and value creation. I'm meeting some fascinating people along the way that have on the ground experience which is important at a time of hype and noise that we're experiencing. Over the coming months, I'm looking forward to sharing these conversations with private equity and those on the AI value chain. Today I'm really looking forward to speaking with Fernanda Dobel, newly appointed Product Director at AI and Chat Clio, a high growth fintech company on a mission to change the world's relationship with money.
SPEAKER_01Yeah, it's great to be here, Steve.
SPEAKER_00Well let let's um let's get a bit of context first, if that's alright. If you could give me a little bit of context on yourself and um Cleo, but obviously in the context and of your experience with with AI.
SPEAKER_01Yeah, yeah. So I um I can definitely give a bit of an intro. So I'm I'm originally Brazilian, despite my American accent, um, but really cut my teeth in in the US and New York where I started my career, um, have been in technology for about 10 years now, uh, starting with education technology out of the States and kind of saw really that grow. Um, and then more recently moved moved to the UK about five years ago, spent um kind of the early pandemic working on a startup called BioCare in the healthcare space, building a clinic to support women going through menopause. And um, and then after that, made my way through the intersection of AI, healthcare, and education uh at a company called Battel, where I was VP of product. And we had a very interesting journey where we brought to market one of the first D2C AI assistants that was that were LLM powered. We helped parents uh sleep train their kids. Um, so built an assistant that could be there, you know, in the wee hours of the morning when parents really need that emotional support and knowing what to do next. And and then most recently I've joined Cleo where I'm product director for chat and AI. I've been there for just over two months, so it's a recent appointment, uh, but we're focusing on how chat um can contribute to Cleo's mission of changing the world's relationship with money, and and yeah, I can definitely talk a little bit more about Cleo there.
SPEAKER_00Yeah, look, um really interesting experience. We've obviously talked for a number of weeks or months now, and um and I was very intrigued by the experience of Patel being a dad, a father, and um and sort of wishing that that something like that was around a few years ago, I have to be honest. But you know, I I know that there's some fascinating um results on on that side of things and your experience there, so you know be good to share that in a little while. Um, you know, with with the recent advancements and accessibility of AI that we've all experienced, particularly obviously generative AI. What opportunities have you seen that sort of present for you, your company, and your customers? And obviously, in the particular field that you've been looking at, and I suppose that's a lot of that is chat related.
SPEAKER_01Yes, I I am in the kind of nexus of chat as an interface, and and what I think is a big paradigm shift that we're all having as we learn to engage with these interfaces, and as these interfaces change um to adapt to our needs. I mean, I think we've got we've got three layers there. I think from an individual perspective, there's there's massive opportunity for efficiency gains and um, you know, tools like GPT and summarization tools, and a lot of these things we've seen flooding the market, even simply like the the AI add-on on Slack and being able to catch up or onboarding into a new company, uh, we use Notion and being able to ask Notion AI what is this acronym mean just saved me from having to ask questions or um spend time sort of looking around. I think in some ways Notion AI is kind of the perfect use case because you've got your own sort of inbuilt database. Um but definitely can see the advantages of having it as a almost a thought partner, um, especially for those of us who are a bit more maybe external thinkers and having that back and forth. Uh one of my colleagues says often, you know, it's it's much easier as a human to prune than to generate. And so if you have something to start with, which these tools, they're not there yet at the level that we need them to be necessarily to leave them alone, um, but they're a great starting point and there are efficiency gains there. I think from a from a company and customer perspective, um what we're seeing in the market and what I really believe to be one of the potential wins for AI is the opportunity to make things accessible that were previously too expensive. So, as I said, um Clio's on a mission to really change our relationship with money, and and it's doing that as the world's first AI financial assistant. So, really democratizing that access to personalized financial coaching and insights based on your behaviors, your spending. And if we look at traditionally where private banking or that kind of coaching has lived, it's really only been offered to the 1% versus the 99%, which is who we're hoping to serve. So, what we like to say is that we're building Clio for a mindset versus a traditional demographic of users. Um, so what we mean by this is people who want to do better with their money and who are overwhelmed maybe by the steps, the educational gap that's needed to get there. So we've got some users who you know are fresh out of school in their early 20s, and we've got some who are much older, maybe in their 40s. Um and and across the sort of spec geographic spectrum of the US, they all have varying but related needs when it comes to to managing their financial health.
SPEAKER_00And there, I mean there's a there's a big sort of spectrum there of age and how they behave, particularly on digitally. Uh do we need to wait for people to catch up in the way that they and the way that they sort of behave with these new technologies? Or have you just got to find a way of of I suppose trying to find a formula that that works for for for different uh age groups?
SPEAKER_01I think we think the behavioral piece I mean that's the beauty of chat is that chat in one way or another is is pretty pervasive with the what adoption of across the world, right? We've got WeChat in China, we've got WhatsApp and whole businesses being run off WhatsApp in Latin America, um, and then kind of text messaging, DMing, WhatsApp. Like it's chat is everywhere already. It's more about, in my perspective, the trust building and the relationship building with okay, this tool is good for this purpose. You know, I I often talk about how Googling has become Google Speak has become such a normal thing for us. You want to search something and you're like 300 Fahrenheit 2C, and you expect Google to understand that you're trying to make a recipe, and you've got kind of that SEO-optimized search to the point where the SEO optimized blogs have sort of destroyed search because now you've got to wade through ads and all this SEO speak in order to actually get to the conversion you want or the recipe you want. Um, and in the same way that we've learned how to Google, um, there's almost an unlearning when it comes to interacting with certainly LLMs, which are a lot more based on you know normal, normal language.
SPEAKER_00Yeah, I tot totally agree. Um my background's in in media and information where we've it knows often it can be really niche information, and a user is used to spending quite a bit of time trying to unpick the information you've you've it's taken a long time to create, finding the bit of information that's critical to them. And so they've sort of I suppose that they're yeah, they've found a way of doing that. Um to then un yeah, unlearn that just to go just ask, just ask the question.
SPEAKER_01As a side note, one of my absolute favorite things to ask GPT is to explain things to me like I'm a third grader, like I'm college age, and you know, like I have a PhD and kind of unpick the difference. And um I find that super useful, even when I'm creating my own presentations and understanding the the level of technicality and helping me build metaphors around the communication I'm doing.
SPEAKER_00So that the the the upgrade that we've obviously seen over the last, I mean, I don't know how is it two years? As the I I assume that the well, the natural language and the fact the responses are far superior than they were before. That must have been a real step up in terms of almost game changing and what you can do for your customer.
SPEAKER_01Yeah, I think it's it's definitely got advantages and and disadvantages, right? Um, and I think the the defensibility comes from understanding how you can leverage your data and how you can be truly personalized in a way that, for example, doesn't hallucinate. Uh, if you think of LLMs, they are working with statistical probabilities, and so having an error rate of absolute zero is unlikely to be the case. And so you have to build systems in a way that take that into account. Um, but with that in mind, it definitely has upgraded the experience tremendously. I mean, we see, I think part of the back to your behavioral question, um, a lot of of customers and and people we interact with, and especially parents at Battel, you know, they'd had these terrible experiences with airline rebookings or customer service agents uh in chat. And so the trust just wasn't there, and and then realizing that okay, if you speak in normal speak like you would to a person, the LLM is much better at responding. Um, so yeah, to to your comment, I think there is a relearning there that needs to happen.
SPEAKER_00Uh yeah, the the I suppose the the error tolerance is less in a regulated uh space, isn't it? Um even you know obviously in the uh in the healthcare space as well. Um how I mean that that that's what is satisfactory in terms of error rate, is it just that I mean, like in most of the tools we have now, there's a there's a little disclaimer, isn't there, saying, Oh, it could it could be wrong. Um it's that I mean that can only go so far, can't it? Especially when when chat is is very much a core part of e-value proposition.
SPEAKER_01I think that um it's it doesn't necessarily have to be an all or nothing, and that's the beauty of the technological advancements and and playing with the technology because we are new to it. It is new to everyone, and you don't have to just use OpenAI out of the box, right? You can work with building rules systems, traditional machine learning in combination with LLMs, using APIs to inject data in where you know you need a rules-based approach. And yes, we definitely don't want to hallucinate um someone's child's name or or in the case of Clio, the contents of someone's bank account. Um, so knowing when to pull from what data source and and what kind of LLMs to use and when to not use LLMs is definitely super important.
SPEAKER_00Yeah, yeah, look, we could talk about chat um for some time and maybe maybe we talk again once we perhaps get into a bit more of a uh a use case. Uh well talking about use cases actually is is you know, there's a you know, there's a there's a challenge there, isn't there? Conundrum of identifying use cases that can achieve differentiated differentiated impact, and that could mean you know, a lot of money, investment, etc. Um but potentially prone to errors and long lead times versus you know looking at use cases that are perhaps more incremental and and small amounts of value, and and maybe that's that's one route that you could take whilst you wait for the technology perhaps to improve and reduce the errors. Is there is it one or the other, or is it just it does it depend?
SPEAKER_01Of course it depends, to you. Um But I I think that the the path is the same though. That you for me you always start with the users, the customers. What are their problems? What are their jobs to be done? What are they experiencing today? And then if you think about the problem they have to solve, where is the opportunity to digitize part or all of the customer journey they're already on? Um, and I think that one thing we don't talk about enough is where is it appropriate to have humans in the loop? So if you think about the Gartner hype cycle that we talk a lot about in tech, right? When you have a new technology, immediately you have this peak of inflated expectations. And it's it's happened to AR and VR, it's happened to Web3, it's now happening to AI, you name it, you expect things to be immediately amazing. Um, and because they're not, inevitably you end up in the trough of disillusionment, and eventually you climb back up to the slope of enlightenment and finally reach the plateau of productivity. And I think what we're seeing with AI is we're realizing, oh, okay, we were a little bit inflated in our expectations today. Uh it doesn't mean we won't get there, but as we climb through this disillusionment that I think is we're slowly, I'm seeing getting to the actual exciting piece, which is okay, there's growth here, there's a lot of opportunity. How can we work with the technology as it exists today? And I actually think having humans in the loop is a great way to mitigate for that. Um and realizing, okay, maybe it's not going to be 100% AI all at once, but you can start to have human oversight and reduce some of the high effort that you had before, which requires some creative thinking. So to tangibilize that, at Battel uh we had human sleep guides on chat 24-7, which made for a very expensive product. By creating an internal recommendation system where the LLM was recommending to our sleep guides surfacing summarizations of our user history, it saved them a lot of time. It was still humans interacting with customers, but that was the first step, and already there were you know economic savings to be made there before you get to a place where you're saying, okay, let's hand this tool directly to users without a human in the loop.
SPEAKER_00Yeah, yeah, okay. Um so it's almost you have a you have a greater sort of vision, perhaps, of how AI can can uh deliver on your strategy, but it's a the it's then the the the journey, just like any sort of product, I suppose it's the journey of how you get to that that that point. But I think that whereas maybe in in previous times we would have waited for that that point of of getting to the absolute sort of perfect solution, when it's it's okay to sort of iterate as as you know as you go. And I think that you know we've seen that from some of the biggest technology companies in the world, haven't we? Is that almost in front of our eyes they're iterating because they they can't afford not not to have this technology in the hands of their users.
SPEAKER_01And I think one of the big differences with this technology is how available it is to everyone in a way that pre Web3 part of the challenge with it was that it was hidden behind this wall of expertise for the most part. Whereas anyone can open an open AI account uh and we're seeing kind of Gemini being rolled out, all of these different tools. You're hearing about artists using it, teachers using it. I think it's permeating the culture uh in in a big way that's unignorable, as you say.
SPEAKER_00Yeah, yeah. Um let's move on to to perhaps talking more specifically. I know you you've you've talked a little bit about Patel there, the use case. Um, wonder if you've got another example to share uh the the how AI has has delivered something with with you know clear ROI, which it can be difficult as well right now.
SPEAKER_01Yeah. Yeah, absolutely. I mean, to expand on that, so I spoke about phase one, which was building this internal recommendation system. Um we then moved into phase two, which is how do you build an entire product out of this? How do you make it user-facing? And the way I think about business opportunities is I like to bucket my risks. So I think about what is the desirability from a user perspective, what is the business viability and kind of commercial viability, and then what's the technical feasibility? And from a technical perspective, doing phase one allowed us to understand, okay, what are the strengths, what are the weaknesses? We we know areas where it might hallucinate, areas where we are willing to have the risk appetite for that, areas where we're saying, oh no, we just have programmed responses because in these instances, um, modes of escalation to say, okay, if someone says A, B, or C, we need to escalate to a human because we're not willing to take the risk on the LLM. So we had a better understanding of the technology, uh, and and we did a lot of commercial modeling to understand, okay, what was the business opportunity and appetite for a more affordable version of our product. But our biggest risk then became the usability and desirability from a user perspective. Asking parents to trust their most precious thing in the world with the advice of a machine uh is not an easy task. And and so we we approach that um in in a couple ways. We we ran a few pilots uh and and kind of very user-led. Um, and for us, the unlock was launching a free trial where we let parents really play with uh L was the name of the AI is the name of the AI assistant, and them unlocking and understanding, having that aha moment became uh crucial in having that adoption because I think there is still a mistrust in the market, and that's only something that will be overcome slowly with time.
SPEAKER_00Do you deliver that aha on a plate or do they have to sort? Or is that immediate because they're the expectation is because they're in a moment of need that they are gonna find that pretty quickly?
SPEAKER_01Yeah, it it really varies depending on on the use case and the pain point. Uh I think one of the big insights was well, with every business onboarding is is super important. But the other one is that people don't know what to ask. Uh, back to the Google example, right? We know exactly what to Google, uh, but we don't know what to ask uh an AI agent. We don't know what we don't know. And so suggesting questions and having a really guided first experience based on your profile, one of the things LLMs are very good at is taking into consideration all of the data and characteristics from a user and then being able to reference those. And so we had you know things in onboarding like asking about a child's sleep space or your child's favorite toy bedtime routine, and being able to then lead a user through talking about their very specific um bedtime routine, for example, meant that there was an immediate trust building there. We realized, oh, this is not your generic um sort of customer service tool that's going to send you down one of three paths.
SPEAKER_00So engagement would have been a good metric to see how you know how people are responding to this. But what what are the other success metrics?
SPEAKER_01Yeah, I mean, it's all the classic ones in engagement, conversion, uh looking at tracking user initiated versus messages we initiated, so through things like push notifications where where the origin is coming from. Um one one thing that was very Very useful as well. Is we had a content piece where parents would read up on our sleep training method, and uh there was a lot of text. We also did text-to-speech so they could listen to this content, but the content itself was only personalized. The you know, the content was personalized based on what paragraphs they saw. Um, but there was a limit to that fitting into your bedroom bedtime routine exactly, for example. And so we added um cards and prompts onto the content that would nudge users uh to use chat to personalize the content. So you've read the content, you've learned about this thing. Okay, now let our AI assistant talk you through how that applies to your specific situations, really creating a bridge, uh, which also from an educational perspective creates that reinforcement loop, which is very nice uh for learning.
SPEAKER_00Yeah, it can be hard, can't it? You know, what whatever situation you're in, you read a book and you go, okay, now I've got to try and try and link that back to my own situation. So I can see how valuable that can be. From a from a business point of view, what was that what was that sort of breakthrough enabling you to do?
SPEAKER_01Achieve. It comes back, it comes back to affordability and and the market size you can really achieve with a tool that's more affordable. Uh it's the same thing at Clio. We're looking at making financial coaching accessible in a world where traditionally it hasn't been. And again, it's, you know, we have all these neo banks that show your transactions, maybe they've categorized them, uh, but there's a big so what? What do I do with this information? There's a lot of anxiety around finances, and and so really what's missing is that coaching element. Okay, but how does that change my behavior next month? What do I do with uh with this now? Which which I think is where the the coaching aspects uh of LLMs can be can be really exciting.
SPEAKER_00Yes, how much are you spending on coffee a month? You sure you can't do without it six times a day? Yeah, yeah. I can I can see how how valuable that can be. Um so what comes next? You know, when you've when you've got that far, um, so you've maybe opened up the market, you've you've been able to get more engagement, the conversions are there. What what what comes next? How do you evolve from there?
SPEAKER_01So I think we're gonna broadly see more and more AI agents and specialization. We're already seeing specialization of some models, uh, we're seeing models that need fewer data to work just as well. And so I imagine what we'll see is uh greater proliferation of these assistants in the market as well as it becoming more normalized from a user experience point of view.
SPEAKER_00Yeah, I think that area is really fascinating. But I mean I listened listened to a podcast the other day from uh Mustafa Sullivan and and he was just very taking a simple use case and just saying how hard the amount of things that need to be done, decisions that need to be done. I think it was booking a a restaurant or something, and and he just was going through the process of you've got to communicate with the different people you want to meet with, uh, then you've got to try and work out what what restaurant and inquire the restaurant. There's so many different steps that it's it's not as simple, you know.
SPEAKER_01We take it for granted as humans, but you know, to do it well, it it's we are as humans, we are definitely the most sophisticated machine out there, yeah. And we we take that for granted for sure.
SPEAKER_00Yeah, yeah, yeah. Uh well look, I I'm really enjoying the conversation. We're coming near to the end, but I do have a couple of questions, more like quick fire ones for you. I think it's really important for us to try and uncover the truth around AI. And whilst at the start we were probably drowning in information from all sorts of sources, so it's always good to hear um from those that I speak with is who do they go to for their sort of source of truth when it comes to AI. So it'd be good to know that from from you.
SPEAKER_01Yeah, um, a lot of crowdsourcing. So through Slack, uh there's a few WhatsApp groups. Um, in terms of newsletters, I'm a big fan of the Neuron. They are a US-based newsletter and they make it very accessible uh to talk about the latest trends. Uh, and then TLDR AI is another one I like looking at.
SPEAKER_00Yeah, Neuron, you got me onto reading those guys. And yeah, that's definitely one of the the clicks I I use. And they seem to be doing other things like uh using AI to create templates and things like that. So, yeah, quite quite interesting. So when I'm Googling now, I'm finding their AI templates. Um, what about a tool that that you're using right now? Um anything uh be good to share with everyone?
SPEAKER_01Yeah, um, I mean classic GPT, which I'm using most days. Uh but the other one that I that has been really interesting for me is uh I found a tutorial online to build this, but I I built so I have this problem that I'm an external thinker. I like thinking out loud and verbalizing, uh which means I talk to myself sometimes, and um or I I work with a lot of voice notes, and I found this tool to essentially build a way to upload a voice note from my phone into OpenAI's whisper and then with an LLM summarize it in Notion. So I basically can talk into a voice note and then I get a summary and a transcript in Notion uh of my ramblings, which um believe it or not are occasionally useful. I wouldn't show them to anyone else, but for me they they work.
SPEAKER_00And and Notion becomes that then that library that's your own um well mini language model, I suppose, doesn't it?
SPEAKER_01Correct, yeah, exactly.
SPEAKER_00How are you using that to any degree at the moment, or that sort of information that you've created? I mean it all depends on using a platform for a period of time, doesn't it? So you've collated it enough for it to be of value.
SPEAKER_01Yeah, I haven't uh I would say that the Notion AI, I haven't used it on my personal notion as much, but but it's been very useful at a company level.
SPEAKER_00Yeah, yeah, okay. Look, we have come to time. Um we could have spoken for for another hour, I'm sure. Um, but we are um hearing from you again. So I'm hosting um breakfast briefings every quarter um for private equity, and very privileged to have Fernanda to uh speak and provide a case study to uh to our audience there on the 17th of September. So really looking forward to that. Thank you for for coming along. Um for those listening, we're providing actual insights, strategies, and tools to turn AI really from a concept to competitive advantage, and you can already hear from Fernandez Fernandez um um words today is that this she's definitely done that. So we've got a a number of P firms signed up, like 3i, Inflection, Montague, ECI, Baumark, Puma, etc. And we have a number of subject matter experts in the room as well, including Fernandez. So you've got a lot of interesting people there to to network with and learn from.