November 16, 2020

Depth and AI with Brandon Gilles and Brian Weinstein

1 hour 41 minutes

Welcome Brandon Gilles and Brian Weinstein of Luxonis Holding Corporation

They make the DepthAI system, which uses the Intel Movidius Myriad X

It was initially created for life safety problems, specifically targeted at distracted drivers.

There is research that a car horn on a bike can help to lower fatal interactions.

Brandon had multiple people in his life hurt or killed by cars while biking.

Though a systemic change is needed, a tech solution is more likely in the short term.

They built a hardware protoype and spent a couple hours of coding to prove out the concept.

"Depth perception + CV is a cheat code", but it needs to be useful in the physical world

No way to embed this much power

Current depth cameras on the market

Intel D435

Intel D455

Two camera solution senses the disparity in the images

AI (OpenCV) helps tell what is there (Car, bus, etc)

Intel compute sticks

Combining depth plus AI

There are other Myriad X SOMs

Jay Carlson episode where he was talking about SOMs and MIPI

2 lane vs 4 lane MIPI

Base boards open source on github

Luxonis did a Kickstarter with OpenCV and raised $1.4M

Working with vendors

Niche that didn't exist: embedded, low cost, fast boot, performant

OpenCV is a compiled library

PyTorch

DNN Module

DARPA grand challenge

OpenCV is a huge community

The DepthAI has bindings for ROS/Python

What is the Myriad X chip background?

Intel bought the Movidius team in 2016 (Vox)

28 processors on the die, with a "Network on chip architecture"

What gets access to the data from the camera first?

It is flexible, so the software configures it

Asymmetric fell out of favor

Geometries of the boards for the Kickstarter

0.4mm pitch, 400 pin BGA

OAK-1 has the color camera, OAK-D has the color camera plus stereo vision

December delivery, which they are on time for

Possible to get the boards right now at higher prices

They are working with Arducam to try and source different camera modules.

Lower cost overall will come from lower cost cameras

Camera module industry is very opaque

This is why there is a custom RPi camera

IMX 378

Apple and Android are doing visual depth preception on phones, including Apple's new LiDAR on the iPhone

Luxonis did a competition around spatial AI and got over 300 entries

Many of the entries involved "giving another sense" to the visually impared.

Following the bald referee

Privacy centric

What is edge computing?

"Embedded is barely a thing yet for AI"

OpenMV by Kwabena Agyeman works on a ST part

They are working on building out the team.

If you're a PCB layout engineer proficient in Altium, send them an email. (remember: The Amp Hour always recommends sharing a portfolio of your work)

SOM Info, including the pinout of the connector on the SOM, is on the github

For more information, check out Luxonis.com

Many thanks to our Patrons! You can join at Patreon.com/TheAmpHour if you’d like to join the crowd. A special thanks to our corporate sponsor Binho, who now distribute the Sensepeek PCBite.

...more

View all episodes

By The Amp Hour (Chris Gammell and David L Jones)

4.9

226226 ratings