Everything Bagel: How ZKPs Are Scaling AI with Privacy & Ownership with Founder, Bidhan Roy

0:06 Welcome to the index podcast hosted by Alex Cahaya. Plug in as we explore new frontiers with entrepreneurs, builders and investors, shaping another episode of Everything Bagel. I'm your host, alex Cahaya, with my co-host Bidon Roy, the founder of Bagel, co-host Bidhan Roy, the founder of Bagel, and today we are going to be focused on what zero-knowledge proofs are and their significance in machine learning, especially in the context of monetizable open source AI.

0:54 Bidhan, do you want to kick it off and maybe just talk about the recent announcement that you guys made about the zero-knowledge proof, the zero-knowledge verification of LoRa training? Let's talk about that, the zero knowledge verification of LoRa training. Let's talk about that. Yeah, yeah, happy to. So we recently made a breakthrough in research internally, which we open sourced. That's what Alex is talking about. But before going into that, I want to take a step back and kind of set the stage on what this means. Then I can go into the technicals of that. So overall, from a bird's eye view, what we do at Bagel is we make open source AI monetizable, and what that means is that we allow open source developers to contribute to open source models.

1:35 That means like training model parameters, and then they can earn the portion of the future revenue of that model in different ways. And one thing that's very important to make open source AI monetizable in this way is that we have to preserve the ownership of all of those contributions that are being made by open source developers. And what do we mean by ownership? By ownership we mean that they have full control of those resources that they contribute. In our specific case, this is some model parameter that they contribute, and how can they have ownership and control of their resources if they don't have any privacy?

2:15 Because if it's open how it works today in traditional open source if it's open for the whole world. Anyone can copy it and run it locally or deploy it in their system. There's nothing stopping them. So the open source developers do not have any control over it. So one of the first things and one of the biggest problems that we had to solve is allow for the open source contributions private but at the same time we give away for the open source developers to prove that they actually contributed without revealing their contributions to the world or any other model owner. So that's the kind of the background on why this relates to what we do, and this has traditionally been a big problem, like one of the biggest problems in machine learning space period, because it's really hard to prove your contribution without revealing it.

3:10 There are a lot of things that goes into it. A lot of people have tried using zero knowledge proofs to do that. I'll give like quick one line about zero knowledge proofs are they are exactly what it sounds like Like you can prove something without revealing the details of that thing. So for our case, the zero-knowledge proof will apply to an open-source developer proving that they contributed to a model, but they provide zero knowledge of the actual contribution to the rest of the ecosystem. People might. They might be running like, like trying to fine tune Lama 4 or whatever, or DeepSeek right, these open source models and the fine tuning.

3:53 Can you explain a little bit for people who might not be aware of just like, what the fine tuning actually is? But before you do that, you know they use parameters to fine tune these models, to improve them, and the parameters could be. You could see the source code, the actual code they write to define those parameters, or you could keep them private, like if the source code is available, then any developer can just run it locally on their own copy of Lumbafor without the original creator of those parameters benefiting from them doing that monetarily right. And so this is a way for you to keep the parameters that you're doing closed actually and monetize it but still allow anybody else to leverage the LLM and the improvements and add on to it in the future.

4:34 So it's kind of a middle ground. Exactly so. One thing I'd like to mention that the technology we have developed it's applicable for general purpose training, so you can train models from scratch with that. The first use case that we have built around it is the fine-tuning part, because there's a big market need for that. But the actual technology that we have, we foresee that being used by millions of developers of coming together and building a massive, massive model from scratch. There's that context. But I want to go back to the technology, actual technology that we are. We just open sourced, so we call it ZK-LORA.

5:14 What it does is it allow developers to prove that they develop some kind of plugin, like small parameter set for a model, which are called LoRa. In traditional machine learning, lora stands for low-ranked adaptation of large language models. So they create a LoRa and those developers can send a zero-knowledge proof that they trained this LoRa to me and I can verify and why this is big. So some context on the zero knowledge space in general. Zero knowledge is very expensive. It takes a lot of time to generate zero knowledge proofs and it takes a lot of resources to build the proofs and all of that.

6:00 Yeah, physical compute power, right, like it's a lot of energy, a lot of compute. Not just that, like, even if you throw a lot of energy and compute, even then it takes like hundreds of hours, if not days or weeks sometimes, if you want to do that. And the zero knowledge proofs has been used for model inference before, like if a model has been generated, an inference if it's coming from a certain model, but it has never been used for model training, what we are doing. Why? Because it's very, very slow to use for model training. You cannot use the zero knowledge proofs for proving model training until now.

6:42 But why it wasn't possible model training until now? But why it wasn't possible? Because it's like it take. It is to take like weeks literally to be able to do verified model training, even for very small models, the models you cannot even use like, let alone the state-of-the-art ones. That's where we are and from that we took a leap with our publishing of our research. All of a sudden you can do this ZK Verify training on state-of-the-art open source models under two seconds. Wow, that's incredible. Yeah, thank you. That's a massive, massive leap.

7:22 Can we go into the details a little bit, like just generally speaking, it's really interesting startups in general when they have these kinds of breakthroughs. You know it's often not the team that has like all the money and hundreds of employees. A lot of times it's like the small scrappy startup with a couple of engineers and that's kind of you guys Like yes, you have, you guys raised a pre-seed couple million bucks, but you're like nine people on your team and this kind of discovery is like people have worked on this for a long time, right, like they've tried to figure this out and the best they could do was like thousands of hours. So how did you guys figure this out? It's kind of crazy.

8:03 How did you do that? I would like to say that this is the kind of a deep seek moment. So all the folks who have been working on this problem they have been kind of hammering at it. They are trying to solve, from a certain way, like we, we do not have that many resources or that many people or that many GPUs to run, have that many resources or that many people or that many GPUs to run. We had to be creative with it. We had to go back and we had to think from first principles like what else can we do to make it fast? That's how we came up with a totally different solution and the unique insight we had is, like all the folks who were spending so much money and they are trying to prove it for so long and most of the companies failed to be able to do so, they, all of them, are trying to prove it for so long and most of the companies failed to be able to do so.

8:45 They, all of them, were trying to prove this is a little bit technical they were trying to prove that some compute was used in some gpu to train some model. So they were trying to get a proof from a model trainer that they burned some compute right and that makes sense, like that's the first kind of problem, the way you want to solve it. And people have been trying to make it faster, like again and again forever. And it became a lot faster. It became thousand x faster as well. But even after thousand x acceleration it's still thousands of hours for very small model. It's far away from being useful. But we could not do that.

9:28 We had to think differently. So we thought, like, why we ask people to prove, prove that they burned some compute? That does not make sense. What do you do in traditional machine learning when we train models? Do we ask people like, hey, do we ask open ai that, hey, did you burn some gpu hours to build this model? No, we just use the model and see if the performance is good. Right, that's how it works. And even when they're training the models, a lot of training runs are thrown out of the window just because they do not meet the evaluation that's required. So what the question they're asking is that if the training run met the desired evaluation score from some specific evaluation dataset, that's it.

10:14 So that kind of was the unique insight for us. We decided, instead of asking the provers to prove that if they burnt any compute, we will ask them if they improved the model. Does not matter how much compute they burnt, we don't care. So that's the first insight. And second insight we had is that you do not really prove the whole model. We built this modular architecture for open source models where, instead of being a monolithic thing, all the developers can kind of provide smaller contributions, which kind of stack on top of each other like Lego pieces.

10:58 Which kind of stack on top of each other like Lego pieces, right, and all of a sudden the contributions they have are so much smaller that the proving on top of that is way more efficient than proving on the whole model. So those are the two unique insights we had and that's how we were able to come to a solution where we allow developers to contribute their own Lego pieces and they are proving that they actually improved the model with those Lego pieces. It does not matter if some compute was burnt or not burnt at all. We care about the performance and we ran with it. We figured out how to do it, we did the benchmarks and we were able to get under two second verification time for LAMA 70B, which is the state of the art model.

11:47 Previous ones was like the maximum we have done on LLMs, zkml or whatever was like 1.5 billion or something. Even that took hundreds of hours. There's a similar story I think I might've told you this before of something like this with Instagram. Right, the founders of Instagram used to work at. One of them used to work on the Gmail team at Google and when he was there they kind of knew that they figured out that, like if you started entering in your username your email address, there was like a 90% chance you were going to log in and so they started preloading your account for you as soon as you started typing. And he took that insight over to Instagram and and basically, if you started writing the caption on a photo, they just uploaded the photo to their database for you and so you'd hit submit and be like bam, the photo was there already.

12:37 And people were like blown away because they were like, what kind of CDN are they using? Like there must be some crazy tech, you know, but it was really just a process thing. It wasn't even like they invented some new fast way to upload, you know, data on the internet. Like the pipes didn't get any bigger, they just use statistics. Yeah, it was just a process innovation. That was in hindsight. It it's like you're kind of like, wow, that's so obvious, but it wasn't right, like no one was thinking about it and I just find it interesting, like the path to these insights. There's this forcing function that gets you to this insight. That's super interesting to me. No, definitely. I mean, I can talk from our experience, right, I can talk from our experience right.

13:18 We could have just taken the easy path and, you know, try to kind of bang the hammer on the nail forever and try to squeeze out a little bit of performance. We're like, no, it doesn't make sense. We have to come from the first principle and we believe that we have started a new era in verifiable decentralized training, which is the verifiable part is important because only then these decentralized training providers can monetize their work. Otherwise it's not possible. All the other attempts that has been made on this problem they are decentralized training but they are not ZK verifiable.

13:59 They're like kind of put together with some duct tapes, which is great. But we opened up this new era where we showed the industry where these does not need to be attached together with duct tapes. They can be verified with zero knowledge, proofs at scale, with great latency and with 100% security guarantees. Yeah, you can be super confident. No one's gaming the system, like this contribution actually did improve this model by this much, and then you can use that data to calibrate how much a developer should be compensated when that thing is used. I've actually been really curious about that part, if you can elaborate a little more. So, now that you've got this proof, do you guys have an algorithm that weights these contributions?

14:44 How does the system know my contribution is worth 50 cents and yours is worth 25? So we are going to release more information on how it works. It's really easy to do so. It's possible to benchmark those contributions. That's how they work. Yeah, like once you have the proof, you can benchmark it, you can prove that the benchmark is accurate really is what it is, Exactly, exactly. Yes, Okay, cool. Those are some of our additional proprietary research that we're doing internally, but we're going to open source it as we roll out our protocol. One of the things that I keep thinking about about ZKP in general, but I also think, just like crypto and the investment that's gone into things like distributed systems and trying to decentralize these different networks and stuff, have like other use cases that people might that might be like non-obvious, like what's the impact in AI to AI, with this ability to verify, outside of just being able to prove for monetization purposes?

15:46 Are there other use cases that this technology, this insight that you guys gained about this, could be used for? First of all, I think monetization is the killer use case. So we, as a decentralized AI industry, have been trying to sell verifiable inference or whatever, like lots of other things, to the Web2 users, and they don't care about verifiability, but what they care about is monetization. So if we show them that this is what you need to do to monetize your contributions, then it makes sense for them. If you just tell them, like the technology, like this, is verifiable, you say blah, blah, blah. A lot of people care, the researchers care, but their users don't care.

16:27 Consumers don't care right, that's how we see it. A lot of other things that's possible through this. Like we have built the first product which is leveraging this technology. It's called the bakery and it's blowing up at the moment. We have had more than 14,000 signups on it since it was launched a month ago, so it's actually blowing up. And the second there are some frontier business models that we can enable with this. I would love to talk about them when we are in process of launching them. So I want to make sure I talk about them when we're in process of launching them. So I want to make sure I talk about them when it's public. I want to make sure that they kind of match the time. But there are some frontier business models that we're enabling which was never possible before period.

17:10 So it's not just not research. This research opens up new business models, entirely new, entirely innovative business models. Well, so you guys have the bakery, but you also have your newsletter where you're sharing all of this research, right? You guys have done a ton of research and that newsletter has got like 50,000 subscribers, you know. So who are these like on the subscriber list? I'm curious, like, who are those people and who do you think they are that are reading the blog? And then the same thing with the people using the bakery. Today there's 14,000 subs, or they're not subscribers, they're just users of the bakery product. Like, who are those people? You know what are they doing. There's a big overlap between those two, by the way. So the readers we have on our blog research blog there is just people who are interested in machine learning.

17:55 They want to learn about the technology. Interested in machine learning, they want to learn about the technology they want to learn about. They care about monetizing open source AI. That's why they subscribe to what we say and it's not something an assumption we have made. We did a survey among our 50,000 subscribers and 73% of them gave us the answer that they are actively looking for a solution where they can monetize their open source contributions. So we are like very aligned, mission aligned. Open source developers are like really behind what we do and what we say. So there's that, and some of them convert to the users. So that's how we kind of monetize. We don't. We have actually gotten offers to put ads on our blog for 20 grand or so.

18:38 We're like that's not the way we're going to monetize. We're going to monetize that the readers who happen to be our target customers and know about us. They trust our brand and then they use the products that we build. That's how we gain the trust among them. So the summary of what I just said it's basically kind of the same demographic of users and developers on both our newsletter and the product. So, okay, let's talk about that a little bit more, because you have 14,000 of them that have converted over there now using the bakery. What kinds of things are they cooking up? A lot, a lot of things actually. So we are seeing a lot of agents and we're seeing a lot of, as I said, like some frontier business models that they're exploring.

19:23 I'd love to talk about them as we work with them and release some of them. But there are some frontier business models, like some developers are coming together and they're going against giants not just open AI and stuff. Like they're actually building domain specific models, not just open AI and stuff. Like they're actually building domain specific models which are somewhere between 25 to 50% better than in their specific domain, than the state of the art model. So some people are building some model in a specific domain and let's say it's like some domain. I'm going to give an arbitrary example. It's a domain where the model knows about podcasts, let's say and then that model performs better 25 to 50% better than state of the art, like cloud anthropic, like whatever state of that model there is cloud anthropic, like whatever state of that model there is.

20:12 That model performs 60% better than the state of the art models when it talks about podcasts. Those are the domain specific use cases that we're seeing. What I'm really excited to see and really curious to see how it plays out is where you know, I come in, I fine tune a model with my parameters and then you take that same thing and you fine tune it further and then we both get compensated. Like that, that mixing of that cross pollination between things is going to be really cool to see. Is that something you're already starting to see amongst some of the users? Exactly. Yeah, I mean the best part about the architecture that we have. It's not even like traditional fine-tuning like fine-tuning does not explain the architecture that well, but like the, what's happening here is that they add their modular contribution on top right and all of a sudden, let's say there's a new shiny model came out.

21:04 Traditionally, what happened is that people have to redo the work from scratch on the new base model. But for what we build, all of the contributors already had the Lego piece. They just take it out and put it on top of the new shiny model. They built it for DeepSeek V2. Now V3 came out, they take it out and put it on top, and that reduces cost for them exponentially as well. Yeah, increases speed too. Time to market right, boom. It's like how long does that take somebody to do that switch? Exactly that's what we're enabling with the architecture. The first piece of research that we published is the verifiability part.

21:46 That is revolutionary even today and we think it will open up new era in decentralized, verifiable training. But at the same time, that's just tip of the iceberg of the things that we're working. Well, let's talk about AGI Artificial General Intelligence. One of the things you and I have talked a lot about like the last couple of weeks, is this idea of open intelligence, open intelligence being AGI running on crypto rails this community owned, and I think Bagel has built the core component to that right, especially with the ability to monetize these open source contributions. What do you think are the components, the other components that need to happen or like how can this community actually, like hypothetically, achieve building an AGI?

22:32 Can this community actually, like hypothetically, achieve building an AGI, even if it could? And also like another sub question is like could it be like a point specific thing, like a niche AGI or I guess general intelligence can't be that niche, but I don't know what your perspective is on that or what's what's like actually makes any sense. But I guess mostly I'm concerned about or want to understand, like, what the actual parts of it are that the community can contribute to, to actually make that if we want to see something like open intelligence exist agi, asi like those are broad concepts, right. and if you want an asi as a community, let's say you want an asi which knows about everything, about every domain there is definitely can build it but it's not the most cost effective.

23:12 But as a group, like some community members, you care about building an ASI which has ASI level intelligence for math or ASI level intelligence for machine learning, asi level intelligence for podcasts, but all the other stuff it does not necessarily need to know about. So all of a sudden it's like we're more cost effective and those are the use cases we enable with our first product that we have built, the bakery. So that's the kind of domain-specific ASI thing. Yeah, it'd be really cool to see how fast this snowballs for something Like there's going to be one specific use case that snowballs to like super intelligence level ai.

23:54 Are you seeing any trends where there's like lots of focused attention on you know? It's like you said, agents are one, but it's like a specific domain you're seeing where, like, lots of devs are like working on this and compounding each other's parameters? Is there one area that you guys have insight into? we actually have have a really good, interesting thing cooking. I am so excited to talk about it, but I want to wait until it's there for the community to see, just to keep up the hype, Right. Okay, all right. I am very excited about this. I am too, because I don't even know about it. We talk like daily, so not daily, but we talk a lot. So I'm interested to figure out what that is. It's going to be exciting when it comes out for sure, like how exciting are we talking about?

24:37 Are we talking about a, like an actual super intelligence for a specific use case? so, uh, you have heard the word like cypherpunk, right, and that's used in the web 3, so it's probably the most cypherpunk technology we have seen in the decentralized AI space. So I just want to say that, wow. I mean, that's a big statement. That's a big statement. It is Like some people talk about these things, the thing we're building. They talk about this in the context of science fiction. We make the fiction reality and we're working on it. So it's very excited to bring it out.

25:17 I'm very sure the community is going to rally behind it. They're going to be excited. When, when are we going to hear about this? Soon, soon soon. Come on, dude, you're giving me nothing except for more excitement. We will have it soon. I can definitely promise yes. Very cool. Well, if people want to get involved, like what do you want them to do? How can the community engage in this specific thing? We have been doing quite a few launches recently and there are different ways to participate. The best way to stay on top of what we're doing is our twitter. It's at bagel open. Ai spelling is exactly how it sounds and there's no difference.

26:00 The audience will see a lot of things that are happening. Our open source initiatives, our products, our research and all of them have a way for the community to come and participate. We always have that option with whatever we do. Yeah, that's the best place and yeah, listen to this podcast as well. Yeah, you'll get all the alpha here for sure, or the alpha for the alpha, which is what I feel like we just got For the ZKLora stuff. Repos open, there's code, there's working examples in there, so people can go and take a look, they can play around with it, they can contribute.

26:41 And there might be some future revenue options for the contributors of the open source code of ZKL or if they're interested in doing so. Okay, cool. Well, it's been awesome. I'm excited to see what this alpha is that's coming soon. We look forward to having another episode next week from you, and thanks for taking the time to share more about ZK, laura. Always, always. It's a pleasure and looking forward to next week. Cool man, Talk to you later. Later. You just listened to the Index Podcast with your host, alex Cahaya. If you enjoyed this episode, please subscribe to the show on Apple, spotify or your favorite streaming platform. New episodes are available every Friday.

27:23 Thanks for tuning in. I feel like I'm.