Hacking my way to epic aerial filming with a Phantom

[Neil Mathew]

Hi guys,

I'm Neil - thanks a lot for reading my post! I hope some of you out there are as obsessed with aerial filming as I am because I wanted to get your thoughts on something I've been working on.

I found that flying a drone and doing camera control at the same time was really hard for me, especially while filming something in motion.So I built a small device that plugs into the phantom and automates the gimbal. The box has a super powerful computer that actually takes your camera feed and runs advanced machine vision software to "track" any subject. All you have to do is click on random subjects on a phone or tablet touch screen, set a composition and the drone automatically keeps your subject in frame no matter how you fly.

Here's my site: http://www.vertical.ai/

I've been reaching out to the community to understand whether this is something pilots and film-makers would be interested in and I'd love to get you thoughts on it. Have you had issues filming tracking shots before ? and Do you think Vertical can help ?

Thanks a lot for your feedback guys! Neil

 

[LuvMyTJ]

A very interesting product. Something I would be interested for sure. Signed up for the Beta.

As a side note I also see your about 2.5 hours from me. I used to go to Guelph often in the summer and I have friends who live in Kitchener... small world.

 

[Bryce]

Looks like After Effects real time. I'd be very interested!

 

[BlackOpsTeamster]

Hi guys,

I'm Neil - thanks a lot for reading my post! I hope some of you out there are as obsessed with aerial filming as I am because I wanted to get your thoughts on something I've been working on.

I found that flying a drone and doing camera control at the same time was really hard for me, especially while filming something in motion.So I built a small device that plugs into the phantom and automates the gimbal. The box has a super powerful computer that actually takes your camera feed and runs advanced machine vision software to "track" any subject. All you have to do is click on random subjects on a phone or tablet touch screen, set a composition and the drone automatically keeps your subject in frame no matter how you fly.

Here's my site: http://www.vertical.ai/

I've been reaching out to the community to understand whether this is something pilots and film-makers would be interested in and I'd love to get you thoughts on it. Have you had issues filming tracking shots before ? and Do you think Vertical can help ?

Thanks a lot for your feedback guys! Neil

Initially I was thinking this was some kind of joke or scam but after looking over your website I can see it's no joke. Was also thinking it was a joke because you asked "is this something we'd be interested in". Uhm, in a couple of words.. Hell's Yeah!
How much more about the device can you tell us? Projected Price? Weight? Personally, I'd like to know as much about it as you're willing to share. That's gotta be some pretty hefty algorithms that would require a lot of processing power, no? Can you explain more about the private beta? And oh, BTW, welcome to the forum and thanks you for stopping by Neil!
BOT

 

[snerd]

A hardware solution for POI or Orbit. Interesting. But don't you think the individual companies offerings of their own software solutions might dampen the demand for this? I don't know, educate me. What is the advantage of this over a straight software solution? (Ignoring the fact that software is not available yet!)

 

[Milly]

I'm in.

 

[BlackOpsTeamster]

A hardware solution for POI or Orbit. Interesting. But don't you think the individual companies offerings of their own software solutions might dampen the demand for this? I don't know, educate me. What is the advantage of this over a straight software solution? (Ignoring the fact that software is not available yet!)

From what I can gather it's not "just a software solution" but actually controls the gimbal for tracking. I could be wrong and that's why I'd like to know as much about this as possible.

[Edit] Just noticed the rest of your website. The links are at the bottom to the specs. Checking it all out now. This is exciting! I gotta feeling this puppy ain't gonna be cheap..
BOT

 

[Neil Mathew]

Looks like After Effects real time. I'd be very interested!

Haha. That's so true now that I think about it. Cool. Can you sign up on the site ? Or just shoot me a message at [email protected]

 

[Neil Mathew]

A very interesting product. Something I would be interested for sure. Signed up for the Beta.

As a side note I also see your about 2.5 hours from me. I used to go to Guelph often in the summer and I have friends who live in Kitchener... small world.

Crazy! Yeah I'm usually in Waterloo but I'm spending most of my time in California these days. A lot of film-makers and pilots here! Are you ever down in SF or LA ?

 

[Neil Mathew]

I'm in.

Awesome! Thanks I'd be happy to tell you more about what we're up to. Could you shoot me a msg at [email protected] ? Looking forward to it!

 

[BlackOpsTeamster]

Hey TJ,
Your pretty good on this stuff. Check my logic could ya?
The site say this operates on 2.4Ghz so that would take my P1 out of contention cause it operates on 2.4Ghz right? BUT my FC40 operates on 5.8Ghz so I'd think I'm good to go with the FC40. The site say's this V1 plugs into the standard sBUS control port available in the drone’s autopilot (They have a P2 listed). The FC40 has an sBUS at X2 on the Naza-M V2 so do you think this thing would work on an FC40?
TIA,
BOT

 

[Neil Mathew]

A hardware solution for POI or Orbit. Interesting. But don't you think the individual companies offerings of their own software solutions might dampen the demand for this? I don't know, educate me. What is the advantage of this over a straight software solution? (Ignoring the fact that software is not available yet!)

Thanks for the feedback! and sorry in advance for the rambling below! I get super passionate about this stuff haha.

So the one thing I always point out is that this isn't just a software solution. Running a vision algorithm like this (which we've developed in-house) is a hugely processor intensive operation. You literally need a powerful linux computer and GPU in there to process the video. Most drones just have autopilots in them. Even something like the Solo which claims to have a "smart computer" on board is really peanuts compares to what is needed. We've spent a long time trying to optimize our software to run fast on a small enough processor to enable this.

Now you're right that this is more of a "feature" than a product but I look at this add-on as something for professional film-makers. It actually works with any drone out there. Well at least the most popular ones. We have folks using this on DJI S900s and Freefly MOVIs for example. And the secret here is that this isn't a one trick pony. The hardware is actually extensible to more software and we have tons of new features being developed to turn regular drones into highly controllable filming beasts.

Over time I do think that manufacturers will start realizing the value of this integrated into their own products. But it's going to take a while and by then, we'll be ready. Nothing's stopping us from building a full system around this software and if it comes down to it, we could even sell modular plug and play systems directly to the manufacturers.

For now, we're going to focus on building amazing shots for filmmakers I hope that answers your questions. I'm pretty responsive on my email at [email protected] so just shoot a msg whenever you want to chat more!

 

[Neil Mathew]

From what I can gather it's not "just a software solution" but actually controls the gimbal for tracking. I could be wrong and that's why I'd like to know as much about this as possible.

[Edit] Just noticed the rest of your website. The links are at the bottom to the specs. Checking it all out now. This is exciting! I gotta feeling this puppy ain't gonna be cheap..
BOT

From what I can gather it's not "just a software solution" but actually controls the gimbal for tracking. I could be wrong and that's why I'd like to know as much about this as possible.

[Edit] Just noticed the rest of your website. The links are at the bottom to the specs. Checking it all out now. This is exciting! I gotta feeling this puppy ain't gonna be cheap..
BOT

You're right! Sorry that wasn't clear! This is great feedback by the way. I'm going to make another demo video of the gimbal actually moving and tracking something live. I'll also release an installation video for the phantom!

Haha so obviously we're still testing pricing which is why it's not on the website yet But if you see the tech specs you'll notice that the final product includes:

1. Onboard unit
2. Base station unit (streams 2.4Ghz H.264 video over 300m with super low latency)
3. Android + iOS app
4. All cables and tools needed for installation.

Could you message me at [email protected] ? I'd love to pick your brain a little more

 

[Neil Mathew]

Initially I was thinking this was some kind of joke or scam but after looking over your website I can see it's no joke. Was also thinking it was a joke because you asked "is this something we'd be interested in". Uhm, in a couple of words.. Hell's Yeah!
How much more about the device can you tell us? Projected Price? Weight? Personally, I'd like to know as much about it as you're willing to share. That's gotta be some pretty hefty algorithms that would require a lot of processing power, no? Can you explain more about the private beta? And oh, BTW, welcome to the forum and thanks you for stopping by Neil!
BOT

Thanks a lot!! "Hells Yeah" is what I love to hear haha. Yep. It's running super hefty vision algorithms on there but we're trying to keep the weight around 200g. We're not there yet but my hardware engineer buddy is like a really annoying dietician. He's obsessed with weight loss on the drone

We're still testing price to be honest. But based on the cost of goods I would say that it's going to be somewhere in the range of $900 - $1400. I think of it as a DJI light-bridge, which costs approximately $1500, but it's got all our software, a massively powerful GPU and controls your gimbal automatically. Also it works on any gimbal (Phantom, Zenmuse Z15, Ronin, Ronin-M, Freefly, 3DR solo, IRIS). Anything with an RC is fair game because we just plug in through the RC channels!

Could you email me at [email protected]. I'd love to chat more !

 

[dirkclod]

in the range of $900 - $1400.

 

[BlackOpsTeamster]

View attachment 28131

Dirk,
Yeah, there may be sticker shock BUT when one considers everything your getting along with what it's capabilities (and potential) are then it really doesn't seem "too" bad. I mean think about it, plugNplay FPV and subject tracking through gimbal control! Not to mention it sounds like it'll have one nice little CPU in there as well running Linux. Don't think I haven't already thought about the possibilities of having a full fledged computer running Linux on my quad! The hacker in me is going nuts..
BOT

 

[dirkclod]

Dirk,
Yeah, there may be sticker shock BUT when one considers everything your getting along with what it's capabilities (and potential) are then it really doesn't seem "too" bad. I mean think about it, plugNplay FPV and subject tracking through gimbal control! Not to mention it sounds like it'll have one nice little CPU in there as well running Linux. Don't think I haven't already thought about the possibilities of having a full fledged computer running Linux on my quad! The hacker in me is going nuts..
BOT

I know and is cool......but that's a P3P

 

[vgt]

This is only gimbal tracking, correct? Seems like it could be of limited use on the phantom line with vertical movement of the camera only (aside from very limited side to side). 90% of the work will still be on the operator, not the software.

Interesting, though.

 

[Neil Mathew]

This is only gimbal tracking, correct? Seems like it could be of limited use on the phantom line with vertical movement of the camera only (aside from very limited side to side). 90% of the work will still be on the operator, not the software.

Interesting, though.

So the goal is to control camera angles and leave position control in the hands of the pilot. That means I want to control tilt and yaw of the camera for any drone.

The way it works is in the case of a 3 axis gimbal, we can control purely the gimbals tilt and yaw and not touch the drone. However, in the case of a 2D gimbal, like on a phantom, we actually control the gimbal's tilt and the done's yaw. It's a really interesting user experience...almost surreal because the drone yaws itself and feels like it has a mind of it's own when you're filming.

That being said, we have a super easy way to switch between manual and auto modes so you're not tied to any one method.

 

[Neil Mathew]

I know and is cool......but that's a P3P
View attachment 28142

You're right that it may seem a littttle pricey at first glance. But think of it this way, the thing has a 2 Ghz CPU with an Nvidia GPU on there and it's running software that's equivalent to the Google self driving car. And the shots it'll let you create will let you pull off what professional drone cinematographers can do. Those folks charge around $300 / hr for their work. Some of the best ones make around $5000 a day for aerial filming. You might be able to pay off the price of this and more ! But obviously I'm biased so think about it....like this contemplative kangaroo.