WEBVTT 00:00.000 --> 00:15.000 Hi everybody, I'm Richard, but a link is in the end of the video and I'll show you a proposal 00:15.000 --> 00:19.000 on how to enhance flash control in video for Linux 2. 00:19.000 --> 00:26.000 This presentation is covering LED flashes only, so if you're doing xenon, that's not an issue. 00:26.000 --> 00:36.000 It's condensed because it's a lighting talk and I use open flashes, interchangeable terms for this thing. 00:36.000 --> 00:41.000 Why wouldn't a better guy like me mess around with flashes? 00:41.000 --> 00:52.000 It's because we're building autonomous driving logistics shuttles, which need to be localized and fine-positioned in a dark warehouse, so no artificial lighting. 00:52.000 --> 01:03.000 Those shuttles drive on rails, those rails have 2D barcoats and engraved, and we have two global shutter cameras on this shuttle. 01:03.000 --> 01:12.000 On the shuttle, which run at 120 frames per second, they have a flash attached, so we can light those barcoats. 01:12.000 --> 01:17.000 The shuttle is moving at 2.5 meters a second, so we need to keep the exposure time low. 01:17.000 --> 01:22.000 To not get any blurs on the image, and therefore we need flash support. 01:22.000 --> 01:29.000 Of course we need to use a mainline Linux kernel, like everybody does, and therefore we need to get a mainline. 01:30.000 --> 01:34.000 But let's first start with a few basics on flash, what's a flash? 01:34.000 --> 01:38.000 It's the light limit in unit connected to a camera. 01:38.000 --> 01:46.000 It may be some kind of next view, or small PCB with a few LEDs on it, or a camera flashes. 01:46.000 --> 01:52.000 You know it, or even a studio stroke, it doesn't matter for the Linux kernel. 01:52.000 --> 01:58.000 Regarding flash control components from a Linux kernel point of view, we have imaging sensor, 01:58.000 --> 02:06.000 which, as you guess, takes the actual image, also takes care of doing exposure and stuff like that. 02:06.000 --> 02:13.000 Then you have a stroke source, the thing which says, now you need to stroke, now we want some light. 02:13.000 --> 02:17.000 This is often part of the imaging sensor. 02:18.000 --> 02:21.000 Those stroke source signal then goes to the flash controller. 02:21.000 --> 02:29.000 The flash controller consumes the signal, and may add some extra configuration on it, which I will show later. 02:29.000 --> 02:33.000 And finally, you have a flash device that's behind the flash controller. 02:33.000 --> 02:38.000 That's the thing we saw before, the thing actually emitting light. 02:38.000 --> 02:43.000 Regarding timing, just to get you all on the same page. 02:43.000 --> 02:47.000 For a rolling shutter camera that's the normal use case, you have a flash. 02:47.000 --> 02:53.000 If it's the same length as the exposure time, just to get it evenly lighted. 02:53.000 --> 02:59.000 If your flash is a little bit lazy, and as a little bit delay, on startup, you may want to add a flash offset. 02:59.000 --> 03:06.000 So the flash is triggered a few microseconds, and another second, before the actual exposure starts. 03:06.000 --> 03:10.000 So the flash can produce the light and get it out. 03:11.000 --> 03:19.000 You can also do a flash offset, positive flash offset, so the flash files later than the exposure time. 03:19.000 --> 03:23.000 And you may reduce the flash duration to be shorter than the exposure time. 03:23.000 --> 03:35.000 For example, if you have a sensor already configured to the minimum exposure time, and the frame is still too light, or other use cases. 03:35.000 --> 03:47.000 So in Linux 2019, which I guess will be released next week, we configure flashes using video for Linux 2 controls. 03:47.000 --> 03:59.000 The first one is the flash LED mode, so you configure the flash system to be flashing, like, my screen. 03:59.000 --> 04:15.000 You can, you must, you have to select the strobe source, so you have two choices here, a source software, which is an unsynchronized flash, so the software is in charge of figuring the strobe or an external source, so hardware strobe source. 04:15.000 --> 04:26.000 For the anchors synchronized flash control, you enable the strobe using visual tool, a control, and develop using the stop. 04:27.000 --> 04:47.000 The flash controller comes outside, you may configure a flash timeout, that's not the flash duration, because the flash timeout defines the hardware timeout for the flash, so the flash controller turns off the flash, after the given amount of time, just to save the flash from burning, that's the idea behind that. 04:47.000 --> 05:01.000 So if your flash will burn after 5 milliseconds of lighting, because it's barely designed, or it's just the case, this control is for you to just limit the time, the maximum time it is allowed to flash. 05:01.000 --> 05:13.000 And some flash controllers offer the possibility to configure flash intensity, which is mostly in milliamps, so you can control the amount of power going in. 05:13.000 --> 05:22.000 I had a bunch of more controls and status and failure stuff, I don't mention here, because that's not that relevant for this talk. 05:22.000 --> 05:35.000 A flash controller isn't always a complex device, like integrated circuit, which can be configured, it may even be just a transistor switching, the power to your LEDs. 05:35.000 --> 06:03.000 So the proposal I'm doing has done is to introduce a flash stroke output enable signal, so we can tell the stroke source, so most typically a camera sensor, he's now allowed to stroke, to do stroke output based on its configuration, so this does not trigger a flash signal, like the control before in the engine, 06:03.000 --> 06:18.000 it just enables the stroke source to stroke, and it's only valid for external stroke sources, I promise it's not my notebook. 06:18.000 --> 06:46.000 Also, I propose to introduce a flash duration control, which is the time the stroke source should actually be flashing, so in this one may then be overwritten by the flash time out in the flash controller, but it's the configured, it's annoying, it's the configured time, the flash source, the stroke source should be flashing, it's typically implemented by the camera sensor. 06:46.000 --> 07:15.000 And finally, as I might have guessed, because of the previous slides, it's a flash offset, a positive and negative time offset between the start of the exposure time and start of the stroke signal, for positive and negative offset, this may be implemented by a stroke source, when it's part of the camera sensor, because the only one in the system knowing when the exposure starts, beforehand, is the camera sensor, because a flash controller, which is, 07:16.000 --> 07:24.000 behind the camera sensor, logically, is only able to do a delay of the exposure time. 07:24.000 --> 07:38.000 So for visualization purposes, here the two configurable controls, you can shift the flash offset around the exposure time start and configure the duration using flash duration. 07:38.000 --> 07:51.000 What's the status of this? The stroke output enable and the flash duration are submitted, are now in the cleanup branch of the maintainer, Sakari, and hopefully, it's now version 10 of the patch set. 07:51.000 --> 08:07.000 Hopefully, it goes in for the next murder window, so Linux 620 or 70, I have no idea, and the flash offset, that's basically in development or done, and it will submit it to the mailing list as soon as the merge window is over and the flash duration. 08:07.000 --> 08:28.000 If the user interface, the user, the API fits their needs, use it with the great to have a few more flash users in, and we for L2 API, if not, and hence it improve it, you know the deal, or just come talk to me. 08:28.000 --> 08:43.000 And as a few lip camera guys are here, because I guess the next two dogs are lip camera, I have seen discussions on the lip camera mailing list about this, and of last year, so maybe there will also be an implementation, the stack upwards, 08:43.000 --> 08:53.000 I'm certainly a camera, so no time for questions to test, but come and reach out, I'm here, personally, or we're one of those channels. 08:53.000 --> 08:54.000 Thank you.