WEBVTT 00:00.000 --> 00:12.360 All right, let's do this welcome full room, let's cool, I'm Benjamin, and yeah, we're here 00:12.360 --> 00:19.560 to talk about the Zaffer, the good, the bad and the ugly, I guess, he words about myself, 00:19.560 --> 00:21.560 I'm a few disclaimers, I guess. 00:21.560 --> 00:26.200 So, yeah, my name is Benjamin, I'm a developer advocate for the Zaffer project, so I should 00:26.200 --> 00:31.080 be telling you about why it's awesome, except that I also know that there's some limitation, 00:31.080 --> 00:38.400 so let's talk about them, and yeah, a few disclaimers or like a caveat, I'm not necessarily 00:38.400 --> 00:43.160 like a developer, an embedded developer by training, I guess I'm a, some would say I'm 00:43.160 --> 00:47.800 a full stack developer, and I certainly can get my way around Zaffer, I've learned a lot 00:47.800 --> 00:53.560 if I see the site join, and I can feel some of the paint points, so yeah, let's talk about 00:53.560 --> 00:54.560 them. 00:54.560 --> 01:01.600 Zaffer, actually question, because that the answer to that question might actually change 01:01.600 --> 01:05.600 the way I will make this particular presentation, how many of you in the room are actually 01:05.600 --> 01:06.600 using Zaffer? 01:06.600 --> 01:08.600 That's a lot. 01:08.600 --> 01:14.880 Okay, so the idea of this talk is that I know already what are some of the paint 01:14.880 --> 01:19.280 points that you guys are experiencing, or at least I think I know, and I wanted to try and 01:19.280 --> 01:25.440 go to some of them really quick, and hopefully we'll have time for more of your own like 01:25.440 --> 01:31.480 genuine questions and feedback at the end, so I might not have to remind you what Zaffer 01:31.480 --> 01:38.720 is, but yeah, open source, turning 10 years old next year, lots of like really really active 01:38.720 --> 01:46.360 community, we just passed 100,000 commits, sometimes during 2024, thousands of contributors, 01:47.360 --> 01:54.720 every week, like literally tens of new contributors every week, and it's an autos, or is 01:54.720 --> 01:55.720 it? 01:55.720 --> 02:01.600 Like, I mean, a piston on paper, it's called Zaffer Autos, so surely it should be, it is a real 02:01.600 --> 02:02.600 time of bringing system. 02:02.600 --> 02:05.120 In fact, over time, it's room, right? 02:05.120 --> 02:11.640 Like, I guess you could say there's Zaffer, the Autos, and there's Daffer project, which 02:11.680 --> 02:18.640 is a little bit more like Zaffer, when you take Zaffer, it's for more of the things for giving 02:18.640 --> 02:23.120 you some hardware abstraction, maybe too much, some would say, but it's that, certainly 02:23.120 --> 02:27.440 like making you completely independent of the underlying hardware, whether it's, we're 02:27.440 --> 02:32.320 talking like architecture, whether we're talking socks, whether we're talking sensors, like 02:32.320 --> 02:35.320 all that is of suckity doubt. 02:35.320 --> 02:42.040 It's effectively a framework, it's much more than just a real time kernel, it's a framework 02:42.040 --> 02:47.280 that provides you with, we have a state machine framework, we have integrations with LVGL 02:47.280 --> 02:52.160 for, for you to build newies, we have obviously all the connectivity stacks, so that's, 02:52.160 --> 02:56.280 I mean, that's not maybe what people would call an autos, when they think autos, they think, 02:56.280 --> 03:02.640 oh yeah, free autos, scheduler, like really low level stuff, only a few, a few sea files 03:02.680 --> 03:07.320 that I'm going to use to support my goal of a cost programming, Zaffer is more than 03:07.320 --> 03:14.320 there, today you learned, if you didn't know that, people building all sorts of connected 03:14.320 --> 03:22.320 devices, so Gardina, they will do a smart irrigation, and 15.4-based protocols, stuff, 03:22.320 --> 03:27.680 in the background, it's just, just a random image of a winter mind, but investors, like 03:27.720 --> 03:35.040 they run and they use Zaffer and the current protocol of all things to control, like to 03:35.040 --> 03:41.360 do live a real-time monitoring of their winterbikes, and also that's that's the effort 03:41.360 --> 03:42.360 based. 03:42.360 --> 03:50.160 So what's the point of the roasting party, I mean, I think, I know, I've joined the Zaffer project 03:50.200 --> 03:57.120 exactly two years ago, like two of the day, I forced them, 2023 was my, like, I was two 03:57.120 --> 04:02.960 days into the job, I love Zaffer, I hope many of you do as well, but sometimes it's not 04:02.960 --> 04:07.760 perfect, it's grown, it's grown over time, there's some, some pain points, so let's 04:07.760 --> 04:14.240 run and talk to them, try and see maybe what's the reason behind it, if there is one, 04:15.200 --> 04:20.320 and I try to let you see the lights maybe at the end of the tunnel for those of you who 04:20.320 --> 04:26.000 might still experience some of those pain points that are listed here, I mean, it's, 04:26.720 --> 04:34.960 yeah, overheard from reddit, discord, devil preserve this that we ran, you've maybe you think 04:34.960 --> 04:41.280 that some of those are true, right, like Zaffer is too big, like it's an autos, how come I need to 04:41.360 --> 04:48.880 download gigabytes of stuff to blink an LED fair enough, it's slow, it's bloated, it's yet another 04:48.880 --> 04:55.360 hardware abstraction, why did you guys, like, divide three four embedded, like, that doesn't 04:55.360 --> 05:03.440 make sense, that's yet another, yet more overhead that you're putting on us, west, why did you bring 05:03.520 --> 05:10.000 your own, and like, build slash Swiss Army Knife sort of tool, just give me the dumb source, 05:10.000 --> 05:16.080 source files, so yeah, there's, and start thinking hopefully we have a few minutes at the end, 05:16.080 --> 05:22.560 if you have more to add to the list, but yeah Zaffer is too big, maybe it is, the thing is, 05:22.560 --> 05:28.800 one thing that we try and, and do is make sure that the time-to-hello world, let's say, 05:28.800 --> 05:36.960 is as quick as possible, and that you get a fully functional workspace as easily as possible, 05:36.960 --> 05:43.520 and effectively, it's pretty much all that you need to do, use west, and we'll talk a bit more 05:43.520 --> 05:51.440 about west in a minute, but basically, provision of workspace pointed to the Zaffer, the main version, 05:51.440 --> 05:58.480 and the upstream version of Zaffer, and then fetch all the dependencies that are given point 05:58.480 --> 06:05.120 in time, are needed to have this Zaffer, this particular version of Zaffer functional, 06:05.120 --> 06:09.520 it means that it actually is going to pull a lot, we're going to see that, bringing in 06:09.520 --> 06:14.880 SDKs as well, so what we call SDKs is effectively tool chains for all the architectures 06:14.880 --> 06:21.200 that you may consider targeting, but like there's a lot, like you might only care about 06:21.200 --> 06:28.160 expressive, and yet installing those SDKs and those tool chains is effectively going to bring 06:28.160 --> 06:34.800 you XADCX and ARM and whatnot, but then the good news is that once you're here, you can 06:34.800 --> 06:41.040 start building a hell of a world, or more, like we have hundreds of code samples, which will 06:41.040 --> 06:47.760 just work on whatever board you're talking, like we have almost 700 boards in Zaffer that 06:47.840 --> 06:56.320 are supported, so you can compile against those boards, flash, debug, it will just work, 06:56.320 --> 07:03.760 but of course that means the drawback is that, and I actually didn't realize that much, 07:03.760 --> 07:12.800 but it's basically you end up with 15 gigs or close to 15 gigs of on your laptop or on your computer, 07:13.760 --> 07:18.400 because that's a lot, like you get the Zaffer sources, which already is actually quite a lot, 07:18.400 --> 07:24.640 when one gig, and then modules, what are modules? Many things can be like embed TLS or nano-PD, 07:24.640 --> 07:30.560 like we mentioned before, and the big bulk of the modules would be the house from the vendors, 07:30.560 --> 07:37.440 which all the rest of the Zaffer like porting layers are being built on top of, and some 07:37.440 --> 07:45.120 house are like over one gigabyte, and same story for the SDKs, we have an easy way for you to 07:45.120 --> 07:50.320 install those, all those tool chains, but some of them, like you will like you never need them, 07:50.880 --> 07:56.640 it's great to have them, I guess, when you work on Zaffer as a Zaffer maintainer, Zaffer, 07:56.640 --> 08:02.960 like someone who touches all the areas of the kernel, and like, and drivers, etc, like you can 08:02.960 --> 08:09.360 really quickly try and see, oh yeah, like, can I really compile my, like, is my drive on? 08:17.040 --> 08:26.240 And we're back. Yeah, like, you might be happy to have all those SDKs, but maybe you don't 08:26.240 --> 08:31.440 need them, so just a reminder, I guess, it's effectively not like you're not forced to do that, 08:31.440 --> 08:37.840 and as a matter of fact, when you build, like, when you start and create a new workspace, 08:38.640 --> 08:42.320 if you follow the Zaffer getting started, yes, you will set up a workspace that effectively 08:42.320 --> 08:49.280 tracks Zaffer upstream, but oftentimes, you will, like, create your own application for which 08:49.280 --> 08:54.880 Zaffer is one of the, one of the dependencies, and when you effectively create the manifest 08:54.880 --> 09:01.200 that describes what it is that you need, you can suddenly allow lists, only those things that 09:01.200 --> 09:06.000 you really know that you're going to ever need, and you don't bring in the how from esteemed 09:06.000 --> 09:12.560 micro, if for the foreseeable future, you only care about NXP, and then you can maybe bring it later, 09:12.560 --> 09:17.120 but that's, yeah, like, that's something for which you can find lots of examples and 09:17.120 --> 09:23.440 best practices in the example application that we maintain in the Git repo, and kind of the same 09:23.520 --> 09:31.520 story recently, we introduced a command that helps provisioning Zaffer SDK into your local environment, 09:32.480 --> 09:39.520 by default, it will install those, I'm actually like, 8 plus gigabytes of tool chains, 09:39.520 --> 09:47.760 but you can easily use the command to be like, I'm on Mac and I care about ARM just bring me 09:47.840 --> 09:54.800 that couple hundred megabytes of tool chain. So, yeah, I guess, I hope that helps with the, like, 09:54.800 --> 10:00.320 yeah, this is too big, but there's a reason, like having those 15 gigabytes that do 10:01.280 --> 10:05.600 download once, you might actually not read it in the end, because you have, you know, that you have 10:05.600 --> 10:12.800 an environment that's already set up to do anything you might ever want to do with Zaffer, so yeah, 10:12.960 --> 10:19.120 Zaffer is bloated, as in it is, yeah, lots of abstracts, however abstractions, a lot of layers, 10:20.320 --> 10:27.040 it's, I want, like, just, sort of almost a bare metal, like really low-level autos 10:27.040 --> 10:35.040 due to real time, because because autos, right? Zaffer is not that, perhaps, but the thing is that, 10:35.040 --> 10:40.160 I mean, Zaffer is, allows you to build, like, really allows you to build really complex applications, 10:40.240 --> 10:46.080 for which you might care about some of those things, maybe not all of them, right? Like, if you're building 10:46.080 --> 10:54.480 some kind of connected Bluetooth battery power, kind of device, surely you're going to care about 10:54.480 --> 10:59.360 having, like, a really good rock solid connectivity stack, you're going to care about power efficiency, 10:59.360 --> 11:06.960 maybe security to some extent, but speed, like, like, as in performance and throughput, 11:07.040 --> 11:17.360 being able to have, like, really intense multitasked environment, maybe not, so you might have, 11:17.360 --> 11:23.840 you might be in that situation, where you actually don't mind having a slightly above-aged 11:23.840 --> 11:30.000 developer experience in that hardware stack protection would be nice to have, like, stack 11:30.080 --> 11:35.440 calories would be nice to have, really nice fancy logs would be nice, because you're, 11:36.480 --> 11:42.400 you don't mind, you kind of forward it, you think it's a good thing in your, for your use case, 11:42.400 --> 11:46.560 and maybe you have something that's completely different, and the thing is that Zaffer allows you 11:46.560 --> 11:53.120 to tweak all those knobs the way you want to, and by default, it takes some reasonable decisions 11:53.120 --> 11:58.080 with regards to what could be a good compromise, something that works for pretty much all use cases, 11:58.080 --> 12:05.760 all sorts of architectures, will run correctly on all the boards that might not be what you need, 12:05.760 --> 12:10.640 but yeah, by default, hardware stack protection is enabled, that might have a small impact on performance, 12:10.640 --> 12:15.280 that also means that should you have a stack overflow and should your hardware platform support it, 12:15.280 --> 12:20.800 you will have, like, a hot fold when something funny was just going, was just about to happen, 12:20.800 --> 12:25.120 because you actually had stack overflow. By default, we optimized for size, not first speed, 12:25.120 --> 12:29.600 so, something to be aware of, but it's just one cake on figure way to be, to be changed, 12:30.560 --> 12:35.200 which if you don't do a lot of defensive programming in the kernel for performance reasons, 12:35.200 --> 12:41.120 but still, there's a few places whereby default, like allocating memory from a memory slot, 12:41.120 --> 12:45.280 like by default, it would be thinking a few extra checks, you might want to disable that, 12:45.280 --> 12:49.440 if you really, really care about the performance, maybe you don't, because you rather have the, 12:49.520 --> 12:56.160 sort of, like, the safety net that it provides you. So, I think, I guess, all that you say, 12:56.160 --> 13:02.400 and kind of related, when it comes to selecting an autos, like, there's many things to look at, 13:02.400 --> 13:08.400 and performance as in, like, I want something that's, like, real time as in fast and high throughput, 13:10.240 --> 13:15.680 first, like, make sure that this is actually video requirement, it might not be or, or even if it is, 13:15.760 --> 13:21.920 it's just, like, one axis among the many that I was showing on, on the radar chart earlier, 13:23.120 --> 13:28.320 and in any case, like, out of the box's effort might not be fine tuned for performance, 13:28.320 --> 13:33.280 and especially, like, benchmark wise, there's always, you need to fine tune for benchmark, 13:33.280 --> 13:40.080 specifically, which, by the way, we've been working on, so there's, there's a lot of benchmarking 13:40.080 --> 13:45.920 tool and, and code in the in the Zephyr repo, but there's one that historically comes from, 13:45.920 --> 13:51.680 from Threatex, that's now been pointed to, to, to Zephyr, which has been a good opportunity for us to, 13:51.680 --> 13:58.720 actually identify a few areas where we were, like, maybe not being optimized, so you can actually 13:58.720 --> 14:04.800 rent those numbers by yourself, and we're not pretty much parity with, with, like, equipped Streatex, 14:05.520 --> 14:10.640 but yeah, like, again, think about this radar chart from earlier, and I think all the accesses there, 14:11.280 --> 14:18.880 matter. Yeah, device 3 device 3 for embedded, that's thousand indexes, that's, again, adding 14:18.880 --> 14:30.560 too much complexity, too much bloat, why, how, why do we use device 4 in Zephyr, while describing the 14:30.560 --> 14:35.280 hardware, that's for sure, providing the hardware initial configuration, whatever, it is that you 14:35.280 --> 14:41.120 configure in your, like, when you configure the, the node corresponding to an LCD display to a nice 14:41.120 --> 14:46.880 spreadsheet bus, whatever, like, all those parameters in terms of speed, resolution, call-off format, 14:46.880 --> 14:52.560 whatever, this is what's going to end up being a past to your drivers, so that there are initialized, 14:52.560 --> 14:58.720 and they have the, like, the right configuration to function, and, yeah, and, and, and effectively, 14:58.720 --> 15:03.440 you end up just, like, just, like, a Linux describing, describing your hardware, and then, 15:03.440 --> 15:10.320 based on what, uh, compatibles are being indicated, and, and, and, and cared for for each node, 15:10.320 --> 15:16.320 then the build system will figure out, like, what is the, uh, what is the, is there a driver available 15:16.320 --> 15:22.480 for this particular compatible, and then, uh, past the data, uh, to it, and, and, and, and, and configure it, 15:22.480 --> 15:28.640 which means that basically there's a lot of stuff happening, um, a build time, where we take all 15:28.640 --> 15:34.160 the metadata, if you will, uh, from, uh, from the device tree, and turn that into, 15:34.160 --> 15:40.080 lots of macros, really, that, that end up, uh, being what's needed to feed, uh, to feed data, 15:40.080 --> 15:46.000 into, uh, and, and, and, and configuration into the drivers, and also to expose the, um, 15:46.000 --> 15:52.400 uh, what, the developers will need when, uh, at, at runtime when they need to basically access, 15:52.400 --> 15:56.480 and get a handle to a particular GPIO, so that they can toggle it, or they need to, 15:56.560 --> 16:01.440 to get a reference to, to a display that, that's works, but that's really, like, a build time thing. 16:01.440 --> 16:07.040 We, we, we generate all that, and, uh, when it works, it's pretty awesome, like, you end up as a developer 16:07.040 --> 16:16.560 to be like, a, I want a display, like, hopefully, like, my, um, my, um, my device supports display. 16:16.560 --> 16:22.160 If yes, um, it will have indicated, uh, in the device tree that there's a chosen node that 16:22.240 --> 16:28.000 points to a particular, um, instance of a display, and then, that means that it can display stuff. 16:28.000 --> 16:32.560 This code, and I actually wanted to bring the demo, but, um, the point here is that, like, the, 16:32.560 --> 16:37.440 the client code, the C code corresponding to that, couldn't care less about the fact that the display 16:37.440 --> 16:44.560 is effectively an LED strip. It's just an LVGL application that only, that only knows that it 16:44.560 --> 16:50.880 runs, uh, it's talking to a bit, a display that's 16 by 16 pixels, and it's just, it just uses it, 16:50.880 --> 16:58.640 and, and if you want to switch, uh, from this to, um, a simulated device running on your on your Linux 16:58.640 --> 17:04.240 machine, it would work just the same, this, the really, the very same code will display the very 17:04.240 --> 17:09.520 same numbers, except that it will be running in an SDL window on your, um, you know, machine. 17:09.520 --> 17:14.000 So that's, um, that's good. When it works, the theory is quite nice, like, you end up, uh, 17:14.000 --> 17:20.720 using all the labels, all the aliases configured in your device tree to get handles and references 17:20.720 --> 17:25.600 to the, the various nodes, in practice, sometimes it doesn't work, and this is something that's 17:26.800 --> 17:32.000 painful confusing for people. The first time this happens to you, and maybe this has happened to 17:32.000 --> 17:37.360 you in the past, like, just like, really go to the bottom, uh, to the bottom of it to understand what's 17:37.360 --> 17:43.440 going on, um, uh, under the hood, or, uh, and, or refer to the documentation to get some, 17:43.440 --> 17:47.920 some tips and tricks, but basically, the error message itself will be really painful to the 17:48.880 --> 17:54.320 server, but, uh, in practice, like, if, if the error happens really at compile time, it's, like, 17:54.640 --> 17:59.680 that you've been, like, the, the node that you're trying to, to basically, uh, uh, get a reference to, 17:59.680 --> 18:04.560 it basically doesn't exist. So whatever, macro, that should have been generated at build time, 18:04.560 --> 18:10.720 just, actually, just doesn't exist. So, um, yeah, it's that, and, or you have a node that requires 18:10.720 --> 18:16.800 a node that itself is disabled. Or maybe that, um, uh, the issue, I'd happen slightly later, uh, 18:18.800 --> 18:24.560 during the linking time, uh, and in which case, probably your device free is all right, but your, um, 18:24.560 --> 18:31.040 your effectively, um, um, um, you might have some drivers that are not, um, um, um, uh, like, uh, 18:31.040 --> 18:37.840 the, the example earlier, I have an LED display, uh, for which, when I configure it, the LED 18:37.840 --> 18:44.160 matrix, it requires an LED strip. And so I might have done everything right there, uh, but if 18:44.160 --> 18:50.880 I didn't enable the LED strip driver, then the LED matrix, driver, uh, I will, uh, we'll have some 18:50.880 --> 18:58.160 issues and, uh, linking what just don't work. Um, West, something that's actually, like, a really 18:58.160 --> 19:03.280 really nice tool in the effort that makes Zephyr, uh, like, when you use West, you have this sort of 19:03.360 --> 19:09.680 single pane of glass to, um, to interact with your devices, like, when you want to, to flash something, 19:09.680 --> 19:14.800 you don't have to remember, uh, whether it's like your flashing for any SP32, what is the Python 19:14.800 --> 19:20.960 command that I can, I should call, uh, or STM or whatever, what is the, uh, um, the command to, uh, 19:20.960 --> 19:25.600 to debug, to, to build, like, this, this takes care of all of that, takes care of, um, managing 19:25.600 --> 19:31.200 all your dependencies. You have, like, this manifest that I referenced, um, reference earlier, which allows 19:31.680 --> 19:37.360 it to, to, um, basically when you have a project that the, the build system and the command line 19:37.360 --> 19:43.280 are very much aware of all the things that make up your, um, uh, your system, and it knows what 19:43.280 --> 19:48.560 board your targeting and sort of, like, what, um, you can have, like, really nice content as Zephyr, 19:48.560 --> 19:54.960 so it's, it's, it's quite nice, um, it's quite extensible as well, uh, some of those, uh, sort of, like, 19:55.440 --> 20:00.960 things that I, I am animating here, um, maybe you're not familiar with all of them, so check out the 20:00.960 --> 20:07.040 documentation that you can use, like, special, uh, build targets in West to, again, in a very 20:07.040 --> 20:14.000 consistent way no matter whether you're targeting arm, intel, STM, espricef, whatever, uh, getting, um, 20:14.000 --> 20:20.480 uh, ram usage report for your application, West build dash key, ram report, you get that, uh, 20:20.480 --> 20:25.840 generating a, uh, software build of material, running code checker, uh, all that is available through 20:25.920 --> 20:32.480 West, so some people don't kind of like, uh, more like, yeah, bring me the, uh, the couple of source files 20:32.480 --> 20:38.000 that make up your autos, make up your kernel, and I'll, uh, and I'll have some fun with it. 20:38.000 --> 20:43.760 I'd rather myself have something that, that gives me something that's, uh, a more, more integrated. 20:43.760 --> 20:49.120 If you don't want to use West, that's fine. Uh, you might have to provide more, uh, information manually 20:49.120 --> 20:54.320 that having West doing the job for you by fetching all the metadata already available in the, in the 20:54.400 --> 21:02.080 manifest and elsewhere, but, uh, it's fine, that works, it's supported, uh, but, uh, um, uh, 21:02.080 --> 21:06.640 last but not least, something we hear often in kind of a segue into the, the, the few minutes, we may 21:06.640 --> 21:12.720 have four for discussion. Uh, my board, my sensor is not supported. Um, yeah, I mean, sure, 21:12.720 --> 21:18.880 if your board, your sensor is not supported, if it's, uh, a board or a sensor from one of those vendors 21:18.960 --> 21:25.280 really engaged, uh, uh, uh, with, with the project, not them, maybe, uh, but if it's just like a 21:25.280 --> 21:29.840 random board random sensor, it's an open source project. It might be a good opportunity for you 21:29.840 --> 21:35.360 to make your first contribution. One thing though, uh, though, um, when we get this kind of feedback, 21:35.360 --> 21:41.440 sometimes it's also along the lines of, yeah, this board or this sensor is supported, but, uh, 21:41.440 --> 21:47.200 it looks still, like it's, it hasn't, uh, and how do I know how, um, how well it works, like, and 21:47.200 --> 21:52.000 does it really support all the, all the features of set sensor? And yeah, sometimes it's, 21:52.000 --> 21:56.560 that the support is not necessarily complete. Uh, we don't have, like, good tools for, 21:56.560 --> 22:02.320 for helping you, um, uh, be, besides just, like, look at the get history, uh, to know whether, yeah, 22:02.320 --> 22:06.720 the file hasn't been touched in five years, maybe it's not up to date and, uh, up to the latest 22:06.720 --> 22:11.920 standards in the effort, um, but if you have tips or, like, suggestions on how we could improve that, 22:12.000 --> 22:17.920 to give better visibility on, sort of, like, the maturity of the various, uh, sock, 22:17.920 --> 22:24.240 portings, board, portings, drivers, um, they don't be great, uh, to do here. And, uh, we actually have 22:24.240 --> 22:30.720 one or two minutes for some additional criticisms, maybe. Thanks. 22:42.720 --> 22:49.680 And now one minute. I would say I love the advisory from a user perspective, but if I want to try 22:49.680 --> 22:55.200 a white driver, it's terrible, because then you go into this mark rose, you think about parsing 22:55.200 --> 23:01.760 that. So I'm curious if, like, we as a community could figure out how to make that's error, 23:02.400 --> 23:07.360 reporting of that, uh, mark rose from the advisory better, so people can actually write more 23:07.440 --> 23:13.280 drivers as well. Yeah, I agree. I, and yesterday in my hotel room, I was trying to, I'm sure there's 23:13.280 --> 23:19.040 it, there's, there are ways. Um, um, I think I wrote a driver turning minutes and the device 23:19.040 --> 23:24.400 he grabbed took me one of ours, right? So, yeah, that goes on right, but that doesn't sound 23:24.480 --> 23:31.760 familiar either or so. Yeah. One more, maybe real quick. 23:46.560 --> 23:52.640 Uh, I really like Zephyr a lot, so thank you for your work on it. Um, my question is, 23:53.040 --> 23:58.720 so you're talking about hardware abstraction earlier. Uh, it makes Zephyr makes hardware abstraction 23:58.720 --> 24:04.000 really nice, especially between different microcontrollers and such. Uh, have you noticed vendors 24:04.000 --> 24:10.800 being upset about this type of thing? Because, um, in the chip shortage, it became easier to, um, 24:10.800 --> 24:17.760 basically change out of part. Uh, I, I haven't experienced that. I, I think it's a feature 24:18.240 --> 24:22.560 to some extent. I think like the vendors, they, they embrace it or they, they have to, 24:22.560 --> 24:28.880 I, I thought of the, the deal and that brings them compete on maybe other other stuff and 24:28.880 --> 24:33.920 stuff that there, there might be stuff that will become a commodity in terms of like IP, IP blocks. 24:33.920 --> 24:39.120 They need to compete on what's like really unique to them. Um, yeah. So thanks. 24:39.120 --> 24:41.120 Thank you.