WEBVTT

00:00.000 --> 00:12.000
All right, our next talk is full by the tracking in SlangVR by SlangVR.

00:12.000 --> 00:30.000
Hello, everyone. It's been 15 years since I was presenting anywhere, so sorry.

00:30.000 --> 00:39.000
My name is Erin. I'm from SlangVR. We do that. I will explain more a bit later.

00:39.000 --> 00:49.000
There's a lot of people. I expect not many of you know what is SlangVR because I see a lot of older faces.

00:50.000 --> 01:06.000
So what does SlangVR do? You take a bunch of small records like that. Put your on your body like I have or like my partner on the photo.

01:06.000 --> 01:13.000
And those trigger connect to a server on your computer or your phone or your headset.

01:13.000 --> 01:22.000
I never worked with Google Slash show. We have Wi-Fi or like I do right now.

01:22.000 --> 01:36.000
We cast them to point four gears on goal. And it all feeds each trigger uses inertial measurement units, inertial measurement units to track their own rotation.

01:36.000 --> 01:47.000
And the server does a magic, which then turns you into I'm a girl, which can be used for VR motion capture.

01:47.000 --> 01:56.000
And we hope for more professional motion capture in the future, but for now you can do stuff like that.

01:56.000 --> 02:02.000
And this is a bad tracking quality, but this is mostly due to interference.

02:02.000 --> 02:09.000
So this is the gist of it. How it happened?

02:09.000 --> 02:23.000
I'm a lot to swear here. So here's the slangVR is a testament to the phrase background and find out.

02:23.000 --> 02:37.000
In late 2000, I bought a headset, which cost two, which cost only 250 dollars, which is pretty cheap for a headset, especially when it came out.

02:37.000 --> 02:48.000
And it was cool, I liked it, but I need more immersion. So I was looking for stuff that can track my body to have my avatar move with my body.

02:48.000 --> 03:01.000
And turns out the only good option was to buy Wi-Fi records, which is like house tracker, which was the previous presenter explained, is a whole different system.

03:01.000 --> 03:11.000
The total cost of entry would be $700, which is like three times more than the headset. And I was like, no, I'm not going to do that. Let's see what else is there.

03:11.000 --> 03:23.000
And until there wasn't much in there, it was mostly a few rough open source projects, specifically kidnapped.

03:23.000 --> 03:39.000
But one of them was an app called overdrag, which is Android app, you can install on your phone, put your phone on your waist, and it's going to track the rotation if you're waist.

03:39.000 --> 03:52.000
And it sounds kind of useless, but actually it increases your immersion because you can estimate the position of your heap, which makes your avatar not just move like that.

03:52.000 --> 04:02.000
But you can actually bend and VR, which already gives you a lot more immersion and a lot of your friends in VR can see more of that.

04:02.000 --> 04:19.000
So I joined their server and was like, hey, why don't you need to use the phone for this? It's used as VIFI, it's used as I'm you, I have ASP board line around, I have around the mind view line around, can I just connect it together and see if it works.

04:19.000 --> 04:34.000
And I ask people like, hey, can you give me like a safety check before order more parts to try and in an hour no one responded, but I was like whatever, I'm going to order it.

04:34.000 --> 04:41.000
So yeah, and while stuff was going like one tracker works fine, it was an easy open source project.

04:41.000 --> 04:49.000
So the problem is that only support is one tracker because who would strap more phones to your body?

04:49.000 --> 04:56.000
There's some people did, which is which time you are supports, you can replace any tracker with your phone.

04:56.000 --> 04:59.000
It's not recommended, don't do that.

05:00.000 --> 05:17.000
So I was like, hey, I need to convince the creator of the old drag to support for like 5 points, so you can have track your legs because you need to track each joints since it's only rotation, not position.

05:17.000 --> 05:22.000
And like, hey, it's open source, you can do it yourself.

05:22.000 --> 05:27.000
Well, I did.

05:27.000 --> 05:29.000
Now I'm any mobile phone.

05:29.000 --> 05:40.000
And without everyone was like, hey, that's kind of cool because a full set of those was like 100 bucks.

05:40.000 --> 05:53.000
It's not very precise as why, well, it can be, it is now, but back then it was pretty junk, but people got really excited about it.

05:54.000 --> 06:03.000
And this was the first, not the first choice, but this was what I did.

06:03.000 --> 06:13.000
It's the server written on Java using swing, which is if anyone knows swing, you would understand, it's very awful UX.

06:13.000 --> 06:26.000
It has purple outlines, so it was kind of pretty, but people said, yeah, but did I mention everything is open source.

06:26.000 --> 06:34.000
It was open source from the start, the project started and open source communities from open source up.

06:34.000 --> 06:42.000
So I put everything open source and people that super excited because when you combine open source and VR, you get you attract the most.

06:42.000 --> 06:46.000
Creative engineers in the world, Furies.

06:46.000 --> 06:58.000
And Furies did that, which sadly the project was 640 pixels, so you can see how pretty it is, but it is better than that.

06:58.000 --> 07:10.000
So we got people to rewrite everything, almost everything and redo the whole UX and proper type script in React.

07:10.000 --> 07:18.000
I help people get react and I also get react and I was like, no, we don't use react, let's keep swing, it's so cool.

07:18.000 --> 07:25.000
Turns out it's easier to support react, now we look pretty.

07:25.000 --> 07:31.000
That took a few years, but that was a journey, so everything is super open source.

07:31.000 --> 07:39.000
If you're interested, this is our stack, we have a lot of layers from here.

07:39.000 --> 07:43.000
I'm not sure if you can read it.

07:43.000 --> 07:57.000
So, I would say a request, and all right, so this is funny, it tracks only when they step, because I don't have a headset.

07:57.000 --> 08:12.000
So, we have six different languages in the project, six C, C++, Rust, Java, Kotlin, five, nice.

08:12.000 --> 08:18.000
And it's very close, it's very hard to support, so we are at least removing Java.

08:18.000 --> 08:33.000
Why Java? Because I started coding from Minecraft, and that was my best language when I started this, so it was Java, but Kotlin is better.

08:33.000 --> 08:50.000
Is anyone interested for me to explain that? Okay, so we have our server that runs on the backend, on Kotlin.

08:50.000 --> 08:59.000
And the GUI is actually a separate app, which is a very modern in this world, it's web app.

08:59.000 --> 09:12.000
But it's not trying chromium, at least it runs towering, which is a fancy wrapper to whatever the operating system has, which is web view on Windows and web key to another thing.

09:12.000 --> 09:23.000
Which non-server-fotent on TypeScript and React. So, that's a software, no, no, but that's not all.

09:24.000 --> 09:37.000
You can see the trackers, we have two different, they feed data into the server through one through Wi-Fi, those through USB.

09:37.000 --> 09:44.000
But that's not all, server does the magic and some apps, you know, have to work.

09:44.000 --> 10:03.000
Some apps can talk directly with our server through a few protocols that we support, like VMC, OSC, which is open sound control, which was invented to control lights for stage performance.

10:03.000 --> 10:08.000
But now is used for everything, basically.

10:08.000 --> 10:17.000
And, but VRChat, which is the main VR social app, actually uses custom open sound control.

10:17.000 --> 10:26.000
We also support that, there is also VMC, which is also, which is what I'm using right now, which is also open sound control.

10:26.000 --> 10:40.000
But, it's also different, and then there is our own solar XAR protocol, which based on flat buffers, which actually monada uses from the previous presenter.

10:40.000 --> 10:45.000
So, we can, our server can talk to monada with that.

10:45.000 --> 10:59.000
And, of course, for VR, you may only want to use steam VR, steam VR, which is a wrapper, which is a host on Windows for all the VR applications.

10:59.000 --> 11:10.000
And, we talk to it, we have a separate driver for it, which is written on C++, and also talk through our custom protocol.

11:10.000 --> 11:20.000
And, then, it talks to games, but we also have another app that talks to steam VR again, and talks to our server.

11:20.000 --> 11:29.000
Why, too, because steam VR is weird, it needs the driver and the app level have different data, so we need both of them.

11:29.000 --> 11:31.000
Yeah, it's a lot.

11:32.000 --> 11:39.000
Some of those more maintained than others, some less, but we are trying.

11:39.000 --> 11:46.000
Now, this is possible only because we are super open source from the start.

11:46.000 --> 11:59.000
I alone wouldn't be able to handle it, because running a business like this is a lot of stuff, and I'm okay programmer, not that good.

11:59.000 --> 12:13.000
So, we got a big community of contributors, and everything we do is completely open source under super permissive licenses, like a patch 2.0 MIT, and like all our software.

12:13.000 --> 12:20.000
And, all our hardware is also open source, that's why we have a lot of clones, which is fine.

12:20.000 --> 12:38.000
It's actually helped us at the start, because people were able to make hardware before we released our own, which sounds disastrous for any commercial company, but for us it was a blessing, because they tested our system.

12:38.000 --> 12:46.000
They found that it works, they found the problems with it, they found the problem with our hardware that we almost put into production.

12:46.000 --> 12:54.000
So, we were able to fix it, and that was great thanks to all of them.

12:54.000 --> 13:08.000
We have a new generation of hardware coming, which is based actually on the community design that community originally started, because they're like, hey, this is great, but there are other microcontrollers, there are other IMUs.

13:08.000 --> 13:14.000
That more energy efficient, that more, more precise.

13:14.000 --> 13:25.000
So, they, for them, for their hardware, they added support into our ecosystem, and then we took what they did, and we also built on top of that to make our next generation.

13:25.000 --> 13:31.000
So, it's a nice symbiosis.

13:31.000 --> 13:35.000
Yeah, we have a lot of consumers.

13:35.000 --> 13:44.000
Our GitHub is, you can see, but there's 186 consumers on GitHub, which I think is pretty big.

13:44.000 --> 13:58.000
Half of them probably are translators, because the system we use uses GitHub accounts, but we have a lot of people who contribute, and we are super proud of the nerves that we built,

13:58.000 --> 14:14.000
and get passionate about open source VR, which, great that we had monada before us, because we were very stand with what they do.

14:14.000 --> 14:20.000
And as I said, we are open hardware, so we have a very big DIY community.

14:20.000 --> 14:28.000
So, if you have a microcontroller and IMU right around, you can probably make slime VR trackers.

14:28.000 --> 14:36.000
From whatever you have, like, that's how I started. You saw the ESP board to the start.

14:36.000 --> 14:41.000
Just don't build a bomb, because they have batteries inside people.

14:41.000 --> 14:49.000
No one has been hurt. I hope not that least people have been hurt, not by our trackers.

14:49.000 --> 14:54.000
But it's just a blessing, but it's so beautiful.

14:54.000 --> 15:01.000
Yeah, we have a big DIY community, and they innovate in our own space, and we innovate in their spaces.

15:01.000 --> 15:09.000
So we provide the communication, we provide, we provide, we open our design, and they build a lot of their stuff.

15:09.000 --> 15:17.000
They make it smaller, faster, more energy efficient, and use different IMUs, and we try to support all of that.

15:17.000 --> 15:23.000
So, where to find us our site is slangware.dev, or you can join our Discord slangware.

15:23.000 --> 15:26.000
This is where we hand out mainly.

15:26.000 --> 15:36.000
We just have blue sky in YouTube, or, yeah, if you have anything like business related or collaboration related,

15:36.000 --> 15:42.000
which out of our support, it will get to correct people.

15:42.000 --> 15:48.000
And I move so far from there.

15:48.000 --> 15:52.000
Let me move myself.

15:52.000 --> 15:58.000
Actually, what I can do is that.

15:58.000 --> 16:01.000
Oh, see?

16:01.000 --> 16:08.000
Now, my head is wrong direction, because I was looking wrong.

16:09.000 --> 16:11.000
Let's call it a set.

16:11.000 --> 16:19.000
The one issue with IMU tracking is that if you know I'm used, it's the drift.

16:19.000 --> 16:25.000
So, like rotation, and especially if the right-to-back position is not,

16:25.000 --> 16:29.000
it's like the precision drops a lot over time.

16:29.000 --> 16:33.000
We have a lot of ways to mitigate it.

16:33.000 --> 16:38.000
So, for normal use, we get like 40 to an hour use.

16:38.000 --> 16:41.000
Of course, it's not like house trackers.

16:41.000 --> 16:44.000
So, we have five minutes left.

16:44.000 --> 16:48.000
Which means I'm ready to the questions, but also.

16:48.000 --> 16:51.000
I know there are people from community here.

16:51.000 --> 16:56.000
We do have a burst of a feather room, which check out our Discord.

16:56.000 --> 16:58.000
I forgot to reach one exactly.

16:58.000 --> 17:00.000
It's a new building.

17:00.000 --> 17:04.000
We will be there, so if you want to hang out, come by.

17:04.000 --> 17:06.000
Yes.

17:06.000 --> 17:08.000
Have a good afternoon.

17:08.000 --> 17:10.000
Have you been speaking to that?

17:10.000 --> 17:11.000
Yeah.

17:19.000 --> 17:21.000
Thank you so much for your report.

17:21.000 --> 17:23.000
Three questions.

17:23.000 --> 17:25.000
First one, do you have a quick start guide?

17:25.000 --> 17:28.000
Second one, what is the hardware?

17:28.000 --> 17:30.000
Is it hardware in or something?

17:30.000 --> 17:32.000
And third one is...

17:32.000 --> 17:34.000
I forgot.

17:34.000 --> 17:44.000
Is it possible to have been using for a VR chat or something?

17:44.000 --> 17:45.000
All right.

17:45.000 --> 17:47.000
I'll give you that.

17:47.000 --> 17:49.000
How do you do it?

17:49.000 --> 17:50.000
How?

17:50.000 --> 17:51.000
Okay.

17:51.000 --> 17:52.000
One, one.

17:52.000 --> 17:53.000
Okay, it works.

17:53.000 --> 17:57.000
Yes, we do have a quick start guide for both day-by

17:57.000 --> 17:59.000
and official ones.

17:59.000 --> 18:02.000
Docs.slimeyard.dev is where you start.

18:02.000 --> 18:06.000
Both, if you want to make your slimes or if you want to use...

18:06.000 --> 18:08.000
Find the someone made.

18:08.000 --> 18:13.000
The hardware is for previous generation that uses Wi-Fi.

18:13.000 --> 18:15.000
We use expressive chips.

18:15.000 --> 18:17.000
We support all of them.

18:17.000 --> 18:21.000
It's built on Arduino framework.

18:22.000 --> 18:24.000
But we use platform I.O.

18:24.000 --> 18:26.000
So we don't use Arduino.

18:26.000 --> 18:28.000
We use platform I.O.

18:28.000 --> 18:30.000
But it is Arduino based.

18:30.000 --> 18:31.000
Yes.

18:32.000 --> 18:33.000
Oh, yeah, VR chat.

18:33.000 --> 18:34.000
Of course.

18:34.000 --> 18:38.000
VR chat is our main is literally what it was built for.

18:38.000 --> 18:41.000
What I'm showing right now is actually like...

18:41.000 --> 18:45.000
More experimental use case, which goes mocap mode,

18:45.000 --> 18:47.000
which makes it track without headset

18:47.000 --> 18:50.000
by counting my steps, basically.

18:51.000 --> 18:52.000
So, yeah.

18:52.000 --> 18:53.000
Thank you.

18:53.000 --> 18:55.000
All right, my questions.

18:55.000 --> 18:56.000
Go.

18:59.000 --> 19:01.000
Yeah, thanks for the talk.

19:01.000 --> 19:02.000
Very cool project.

19:02.000 --> 19:06.000
I was wondering, since you benefit so much from the contributions

19:06.000 --> 19:10.000
back from the community, why didn't you pick a copyright license

19:10.000 --> 19:14.000
that enforces companies who would take your stuff

19:14.000 --> 19:17.000
and add something to it to contribute it back as open source.

19:18.000 --> 19:19.000
That's a good question.

19:19.000 --> 19:21.000
I really hate GPL.

19:21.000 --> 19:24.000
And all copyright licenses.

19:24.000 --> 19:27.000
I don't think that free software,

19:27.000 --> 19:30.000
if you will place a burden on it.

19:30.000 --> 19:32.000
I understand why it quite exists.

19:32.000 --> 19:35.000
It's a good thing for like, subject Linux.

19:35.000 --> 19:38.000
But for us, we wanted something easy.

19:38.000 --> 19:42.000
And it's a mixed bag.

19:42.000 --> 19:45.000
Some people give back, some people don't.

19:45.000 --> 19:49.000
We're trying to build the community that based on

19:49.000 --> 19:52.000
sharing on the open source.

19:52.000 --> 19:55.000
So, most people try to do try to give back.

19:55.000 --> 19:59.000
And I think the software should be free in general,

19:59.000 --> 20:02.000
like MIT and Apache is what I personally stand for.

20:02.000 --> 20:04.000
So, that's why it's that.

20:04.000 --> 20:09.000
And a lot of our maintainers also think similar.

20:09.000 --> 20:10.000
All right.

20:10.000 --> 20:13.000
Time for maybe one more question.

20:15.000 --> 20:28.000
Cool.

20:28.000 --> 20:34.000
By the way, and the next speaker, please come over to the station.

20:34.000 --> 20:35.000
Hi.

20:35.000 --> 20:39.000
I see a lot of, like, your demo.

20:39.000 --> 20:43.000
And the presentation was about having a,

20:43.000 --> 20:47.000
I guess, like, a laptop or a server or something nearby.

20:47.000 --> 20:51.000
Has there been any use or any consideration for doing,

20:51.000 --> 20:56.000
like, a mobile or something that you could put in,

20:56.000 --> 20:58.000
like, a backpack or something.

20:58.000 --> 21:03.000
If I wanted to say, like, record myself out, like,

21:03.000 --> 21:06.000
skateboarding or snowboarding or something like that.

21:06.000 --> 21:07.000
I don't want to display.

21:07.000 --> 21:10.000
I just want to record and then play back later.

21:10.000 --> 21:11.000
Yes.

21:11.000 --> 21:14.000
So, you actually can do that already.

21:14.000 --> 21:17.000
The slender server runs on Android.

21:17.000 --> 21:20.000
We do have some plans to get to iOS,

21:20.000 --> 21:23.000
but the ecosystem is very complicated.

21:23.000 --> 21:27.000
And it can run on any, like, Raspberry Pi on,

21:27.000 --> 21:30.000
have set itself for standard sets.

21:30.000 --> 21:35.000
So, you can run slender server on pretty much anything.

21:35.000 --> 21:39.000
And if you have a, you need the Wi-Fi hotspot.

21:39.000 --> 21:43.000
If you use the Wi-Fi slimes, or you need a dongle.

21:43.000 --> 21:44.000
But it works.

21:44.000 --> 21:46.000
And the server itself supports,

21:46.000 --> 21:51.000
recording into a motion capture from at which calls BVH,

21:51.000 --> 21:55.000
which does record all your movements.

21:55.000 --> 21:58.000
It might, it's still experimental,

21:58.000 --> 22:00.000
even though it existed for three years.

22:00.000 --> 22:02.000
Because not many people use it.

22:02.000 --> 22:05.000
But if people are interested in that,

22:05.000 --> 22:08.000
and start using it, there will be more development

22:08.000 --> 22:09.000
than it.

22:09.000 --> 22:14.000
And we are really, we really want to try to push the motion capture of SlangVR

22:14.000 --> 22:16.000
forward more.

22:16.000 --> 22:18.000
So, yeah, you can do that.

22:18.000 --> 22:19.000
All right.

22:19.000 --> 22:20.000
Thanks a lot.

22:20.000 --> 22:22.000
Unfortunately, we are at a time, so you can.

22:22.000 --> 22:24.000
Thank you.