Welcome to PhantomPilots.com

Sign up for a weekly email of the latest drone news & information

Hacking my way to epic aerial filming with a Phantom

Discussion in 'General Discussion' started by Neil Mathew, Aug 21, 2015.

  1. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    Hi guys,

    I'm Neil - thanks a lot for reading my post! I hope some of you out there are as obsessed with aerial filming as I am because I wanted to get your thoughts on something I've been working on.

    I found that flying a drone and doing camera control at the same time was really hard for me, especially while filming something in motion.So I built a small device that plugs into the phantom and automates the gimbal. The box has a super powerful computer that actually takes your camera feed and runs advanced machine vision software to "track" any subject. All you have to do is click on random subjects on a phone or tablet touch screen, set a composition and the drone automatically keeps your subject in frame no matter how you fly.

    Here's my site: http://www.vertical.ai/

    I've been reaching out to the community to understand whether this is something pilots and film-makers would be interested in and I'd love to get you thoughts on it. Have you had issues filming tracking shots before ? and Do you think Vertical can help ?

    Thanks a lot for your feedback guys! :D Neil

    [​IMG]

     
    BlackOpsTeamster likes this.
  2. LuvMyTJ

    LuvMyTJ ADMINISTRATOR
    Staff Member

    Joined:
    Feb 9, 2014
    Messages:
    6,811
    Likes Received:
    4,376
    Location:
    Live! From New York!
    A very interesting product. Something I would be interested for sure. Signed up for the Beta. ;)

    As a side note I also see your about 2.5 hours from me. I used to go to Guelph often in the summer and I have friends who live in Kitchener... small world.
     
  3. Bryce

    Joined:
    Jun 13, 2015
    Messages:
    445
    Likes Received:
    95
    Looks like After Effects real time. I'd be very interested!
     
    Neil Mathew and BlackOpsTeamster like this.
  4. BlackOpsTeamster

    Joined:
    Aug 12, 2014
    Messages:
    295
    Likes Received:
    266
    Location:
    St. Louis metro east
    Initially I was thinking this was some kind of joke or scam but after looking over your website I can see it's no joke. Was also thinking it was a joke because you asked "is this something we'd be interested in". Uhm, in a couple of words.. Hell's Yeah!
    How much more about the device can you tell us? Projected Price? Weight? Personally, I'd like to know as much about it as you're willing to share. That's gotta be some pretty hefty algorithms that would require a lot of processing power, no? Can you explain more about the private beta? And oh, BTW, welcome to the forum and thanks you for stopping by Neil!
    BOT
     
    #4 BlackOpsTeamster, Aug 21, 2015
    Last edited: Aug 21, 2015
    Neil Mathew likes this.
  5. snerd

    Joined:
    May 5, 2015
    Messages:
    2,633
    Likes Received:
    1,585
    Location:
    Oklahoma
    A hardware solution for POI or Orbit. Interesting. But don't you think the individual companies offerings of their own software solutions might dampen the demand for this? I don't know, educate me. What is the advantage of this over a straight software solution? (Ignoring the fact that software is not available yet!)
     
    Neil Mathew likes this.
  6. Milly

    Joined:
    Dec 25, 2014
    Messages:
    122
    Likes Received:
    37
    Location:
    Wick, Caithness, Highlands of Scotland
    I'm in.
     
    Neil Mathew likes this.
  7. BlackOpsTeamster

    Joined:
    Aug 12, 2014
    Messages:
    295
    Likes Received:
    266
    Location:
    St. Louis metro east
    From what I can gather it's not "just a software solution" but actually controls the gimbal for tracking. I could be wrong and that's why I'd like to know as much about this as possible.

    [Edit] Just noticed the rest of your website. The links are at the bottom to the specs. Checking it all out now. This is exciting! I gotta feeling this puppy ain't gonna be cheap.. ;)
    BOT
     
    #7 BlackOpsTeamster, Aug 21, 2015
    Last edited: Aug 21, 2015
    Neil Mathew likes this.
  8. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    Haha. That's so true now that I think about it. Cool. Can you sign up on the site ? Or just shoot me a message at neil@vertical.ai :)
     
    BlackOpsTeamster likes this.
  9. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    Crazy! Yeah I'm usually in Waterloo but I'm spending most of my time in California these days. A lot of film-makers and pilots here! Are you ever down in SF or LA ?
     
    BlackOpsTeamster likes this.
  10. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    Awesome! Thanks :) I'd be happy to tell you more about what we're up to. Could you shoot me a msg at neil@vertical.ai ? Looking forward to it!
     
    BlackOpsTeamster likes this.
  11. BlackOpsTeamster

    Joined:
    Aug 12, 2014
    Messages:
    295
    Likes Received:
    266
    Location:
    St. Louis metro east
    Hey TJ,
    Your pretty good on this stuff. Check my logic could ya?
    The site say this operates on 2.4Ghz so that would take my P1 out of contention cause it operates on 2.4Ghz right? BUT my FC40 operates on 5.8Ghz so I'd think I'm good to go with the FC40. The site say's this V1 plugs into the standard sBUS control port available in the drone’s autopilot (They have a P2 listed). The FC40 has an sBUS at X2 on the Naza-M V2 so do you think this thing would work on an FC40?
    TIA,
    BOT
     
  12. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    Thanks for the feedback! and sorry in advance for the rambling below! I get super passionate about this stuff haha.

    So the one thing I always point out is that this isn't just a software solution. Running a vision algorithm like this (which we've developed in-house) is a hugely processor intensive operation. You literally need a powerful linux computer and GPU in there to process the video. Most drones just have autopilots in them. Even something like the Solo which claims to have a "smart computer" on board is really peanuts compares to what is needed. We've spent a long time trying to optimize our software to run fast on a small enough processor to enable this.

    Now you're right that this is more of a "feature" than a product but I look at this add-on as something for professional film-makers. It actually works with any drone out there. Well at least the most popular ones. We have folks using this on DJI S900s and Freefly MOVIs for example. And the secret here is that this isn't a one trick pony. The hardware is actually extensible to more software and we have tons of new features being developed to turn regular drones into highly controllable filming beasts.

    Over time I do think that manufacturers will start realizing the value of this integrated into their own products. But it's going to take a while and by then, we'll be ready. Nothing's stopping us from building a full system around this software and if it comes down to it, we could even sell modular plug and play systems directly to the manufacturers.

    For now, we're going to focus on building amazing shots for filmmakers :) I hope that answers your questions. I'm pretty responsive on my email at neil@vertical.ai so just shoot a msg whenever you want to chat more!
     
  13. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    You're right! Sorry that wasn't clear! This is great feedback by the way. I'm going to make another demo video of the gimbal actually moving and tracking something live. I'll also release an installation video for the phantom!

    Haha so obviously we're still testing pricing which is why it's not on the website yet ;) But if you see the tech specs you'll notice that the final product includes:

    1. Onboard unit
    2. Base station unit (streams 2.4Ghz H.264 video over 300m with super low latency)
    3. Android + iOS app
    4. All cables and tools needed for installation.

    Could you message me at neil@vertical.ai ? I'd love to pick your brain a little more :)
     
    BlackOpsTeamster likes this.
  14. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    Thanks a lot!! "Hells Yeah" is what I love to hear haha. Yep. It's running super hefty vision algorithms on there but we're trying to keep the weight around 200g. We're not there yet but my hardware engineer buddy is like a really annoying dietician. He's obsessed with weight loss on the drone :)

    We're still testing price to be honest. But based on the cost of goods I would say that it's going to be somewhere in the range of $900 - $1400. I think of it as a DJI light-bridge, which costs approximately $1500, but it's got all our software, a massively powerful GPU and controls your gimbal automatically. Also it works on any gimbal (Phantom, Zenmuse Z15, Ronin, Ronin-M, Freefly, 3DR solo, IRIS). Anything with an RC is fair game because we just plug in through the RC channels!

    Could you email me at neil@vertical.ai. I'd love to chat more !
     
    BlackOpsTeamster and Alex Folds like this.
  15. dirkclod

    dirkclod Moderator
    Staff Member

    Joined:
    May 9, 2014
    Messages:
    10,652
    Likes Received:
    5,705
    Location:
    Amory Mississippi
    fall down.png
     
    GoodnNuff, ianwood, LuvMyTJ and 4 others like this.
  16. BlackOpsTeamster

    Joined:
    Aug 12, 2014
    Messages:
    295
    Likes Received:
    266
    Location:
    St. Louis metro east
    Dirk,
    Yeah, there may be sticker shock BUT when one considers everything your getting along with what it's capabilities (and potential) are then it really doesn't seem "too" bad. I mean think about it, plugNplay FPV and subject tracking through gimbal control! Not to mention it sounds like it'll have one nice little CPU in there as well running Linux. Don't think I haven't already thought about the possibilities of having a full fledged computer running Linux on my quad! The hacker in me is going nuts.. :D:D:D
    BOT
     
    dirkclod and Neil Mathew like this.
  17. dirkclod

    dirkclod Moderator
    Staff Member

    Joined:
    May 9, 2014
    Messages:
    10,652
    Likes Received:
    5,705
    Location:
    Amory Mississippi
    I know and is cool......but that's a P3P :eek:
    fall down.png
     
  18. vgt

    vgt

    Joined:
    Feb 14, 2015
    Messages:
    279
    Likes Received:
    72
    This is only gimbal tracking, correct? Seems like it could be of limited use on the phantom line with vertical movement of the camera only (aside from very limited side to side). 90% of the work will still be on the operator, not the software.

    Interesting, though.
     
  19. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    So the goal is to control camera angles and leave position control in the hands of the pilot. That means I want to control tilt and yaw of the camera for any drone.

    The way it works is in the case of a 3 axis gimbal, we can control purely the gimbals tilt and yaw and not touch the drone. However, in the case of a 2D gimbal, like on a phantom, we actually control the gimbal's tilt and the done's yaw. It's a really interesting user experience...almost surreal because the drone yaws itself and feels like it has a mind of it's own when you're filming.

    That being said, we have a super easy way to switch between manual and auto modes so you're not tied to any one method.
     
    BlackOpsTeamster likes this.
  20. Neil Mathew

    Joined:
    Aug 21, 2015
    Messages:
    421
    Likes Received:
    105
    Location:
    Mountain View, CA
    You're right that it may seem a littttle pricey at first glance. But think of it this way, the thing has a 2 Ghz CPU with an Nvidia GPU on there and it's running software that's equivalent to the Google self driving car. And the shots it'll let you create will let you pull off what professional drone cinematographers can do. Those folks charge around $300 / hr for their work. Some of the best ones make around $5000 a day for aerial filming. You might be able to pay off the price of this and more ! But obviously I'm biased so think about it....like this contemplative kangaroo.

    [​IMG]
     
    dirkclod and BlackOpsTeamster like this.