Welcome to PhantomPilots.com

Sign up for a weekly email of the latest drone news & information

Autonomous camera aiming & tracking module

Discussion in 'General Discussion' started by xtracrispy, Sep 6, 2014.

  1. xtracrispy

    Joined:
    Sep 6, 2014
    Messages:
    7
    Likes Received:
    0
    Hello,

    So I've been playing around with drone videography lately, and quickly realized that it's really tough to aim the camera while flying a quad in anything but a straight line.
    I envisioned this one sweeping pan shot of an object in a field, but just couldn't do it with the quadrotor rolling around due to my flight control.

    Anyway, I'm a roboticist and hacked together a working prototype that uses the video stream to track anything I want, using computer vision algorithms.

    Here's a quick youtube video to show what I mean:
    https://www.youtube.com/watch?v=QJW9ReKU-0I&list=UUy6SYEB_Gk40j9NOy8RI5aQ

    So far, I programmed in a smooth-motion and the ability to track anything I select in a box. In the future I am going to make an android/ipad app so that you can just multi-touch define a subject with 2 fingers.
    After looking into the DJI D-bus, I'm fairly sure I can make the Zenmuse/Ronin gimbals follow my commands, in addition to commanding the flight paths of the quad itself to follow something (ie. a bike, or a car)

    Is anyone out there also interested in this tech? It's easy enough for me to make and share, and any feedback on cool ideas to implement would be great :D
    Cheers
     
  2. ProfessorStein

    Joined:
    Jun 20, 2014
    Messages:
    1,136
    Likes Received:
    3
    Location:
    Portland, OR
    That sounds ultra smart! We've seen drones that autonomously track objects by locking onto a supplemental transceiver (like the Air Dog)... but if you're tracking purely in the video stream, that's a huge step!
     
  3. noiseboy72

    Joined:
    Jul 13, 2014
    Messages:
    575
    Likes Received:
    17
    Location:
    Lincs, UK
    Brilliant! A real market for this I would say. Are you suggesting this would be an add on module for the quadcopter, with WiFi link back to a smart phone to define targets?
     
  4. xtracrispy

    Joined:
    Sep 6, 2014
    Messages:
    7
    Likes Received:
    0
    Yes, what I'm currently thinking is a small block that lifts off with your quadrotor so you get high-speed control without lag.
    A little wifi transmitter can stream a video or picture feed, so that you can tell the system what to look at.

    After that, just fly around in circles and get perfect video!
     
  5. noiseboy72

    Joined:
    Jul 13, 2014
    Messages:
    575
    Likes Received:
    17
    Location:
    Lincs, UK
    Get it built! I am first in the queue, cash in hand :)
     
  6. DKDarkness

    Joined:
    Dec 16, 2013
    Messages:
    554
    Likes Received:
    3
    Location:
    Denmark
    Looks really promising! Keep up the good work :)
     
  7. ramirez

    Joined:
    Aug 29, 2014
    Messages:
    2
    Likes Received:
    0
    Location:
    Hants, UK
    Looks great.
     
  8. Buckaye

    Joined:
    Aug 24, 2014
    Messages:
    1,557
    Likes Received:
    420
    Location:
    Orlando
    wow! That's awesome! Great work!

    Now... I have to ask... you know not to make robots stronger than us right? :)
     
    landed and Guardmy6 like this.
  9. Peter Patricelli

    Joined:
    Sep 11, 2013
    Messages:
    267
    Likes Received:
    0
    Location:
    Eugene, OR USA
    Where you are going with this is the inevitable future of drone videography. Someone has got to be the first, and stay out front on this development, might as well be you.

    But if I might suggest.....

    "it's really tough to aim the camera while flying a quad in anything but a straight line.
    I envisioned this one sweeping pan shot of an object in a field, but just couldn't do it with the quadrotor rolling around due to my flight control"

    You are correct....for normal flying modes. The ORIENTATION of the bird determines where the camera is pointing and for the pilot keeping track of that orientation...determined by the required camera direction....and making the required mental adjustments and switches to keep the bird flying in the desired direction INDEPENDENT of the nose orientation....is VERY difficult.

    Which is EXACTLY why you should look at IOC, specifically Home Lock or Course Lock flying...while doing videography. Suddenly the right hand stick consistently and directly controls the birds 2D movement, direction and speed, completely independent of bird....and camera.... orientation and filming requirement. At the same time the left hand stick is free to direct the camera....and bird orientation...completely independent of the birds flying requirements.

    Seems to me that what IOC flying achieves for videography solves a LOT of the problems you will face. Without it you would need an independent 360 degree 3D gimbal.......and a whole new generation/design of drones.

    Realizing what IOC could do, yesterday I EASILY pulled off a full 360 degree sweeping pan of myself standing in the center (Home Lock) position, something that would have been difficult, and ugly (for me anyway) in regular flying mode. I bet in IOC you could pull off the pan you envisioned in your opening statement.

    Using IOC, you would only have to link your tracking algorithm to the left hand stick YAW control...and you have GOT IT!

    Peter Patricelli
     
  10. xtracrispy

    Joined:
    Sep 6, 2014
    Messages:
    7
    Likes Received:
    0
    Hi Peter,

    Thanks for testing that and providing the very good suggestion! Definitely a good "keep it simple" approach to solving the problem.
    Can you think of a way to get a double-dynamic shot with our current technology also?

    For example, your signature's website has some windsurfing shoots where people are moving over the water and doing jumps. If the subject is moving also, I wonder if there's a way to track it other than via our ground-based eyeballs.
     
  11. Happyflyer

    Joined:
    May 5, 2014
    Messages:
    1,805
    Likes Received:
    98
    Location:
    Cold, Cold, Michigan
    You got that right.
    .
    Great! Keep working.
     
  12. The Editor

    Joined:
    Aug 7, 2013
    Messages:
    568
    Likes Received:
    7
    FPV and a litttle practice or move to a two man rig and a 360 degree gimbal - simples.
     
  13. Peter Patricelli

    Joined:
    Sep 11, 2013
    Messages:
    267
    Likes Received:
    0
    Location:
    Eugene, OR USA
    The programming technology of lock-on auto-focus exists, since my D800 camera in video mode will lock on thefocus of a 600mm lens, very critical, to a moving windsurder against a background of moving waves. From there to centering the locked-on subject ought to be relatively simple, programming-wise.

    I did a test flight, actually filming, of panning independently while flying in HC/Home lock, this morning. My FPV monitor is currently broken, awaiting a replacement being shipped, so this was strictly visual flying. Other than a need to slow down the panning speed, a perpetual problem, the results were very good for a first time, out of the box test. Without HC/Home Lock I could not have come close to the same results.

    Try it!!

    Peter Patricelli
     
  14. MadMitch88

    Joined:
    Aug 19, 2014
    Messages:
    539
    Likes Received:
    10
    Location:
    Cleveland, OH
    God **** you, bro --- get this thing perfected for the Phantom Vision+ and upload it to the Google Play store and then take my money !!! :mrgreen:

    But seriously --- I'm glad so many smart fellas like yourself are getting interested in the UAV hobby because great forward progress like motion tracking video capture will soon be a reality. I can only imagine how many cool uses this will have for everyone who flies and wants to get great shots of a particular object during flight. I know some big game hunters who would love to have an easy way to keep animals in the center of the frame while the UAV buzzes around. Your invention would do that.

    Keep up the good work and I hope to see your hard work pay off with cold, hard cash :D
     
  15. xtracrispy

    Joined:
    Sep 6, 2014
    Messages:
    7
    Likes Received:
    0
    Thanks for the compliments guys! It's really motivating me to try new things now :lol:

    I wonder if that could be a possibility one day. What would you guys think this to be worth, given that the electronics will probably cost a few hundred for parts?
     
  16. fastsmiles

    Joined:
    Apr 22, 2014
    Messages:
    844
    Likes Received:
    130
    Location:
    Fort Worth, Tx
    This is awesome technology, keep us posted on progress. There is a great demand for this ability. What kind of price do you project?
     
  17. yusiang

    Joined:
    Sep 7, 2014
    Messages:
    2
    Likes Received:
    0
    I think a similar setup(non-DJI) has done it with some off-the-shelf hardware under $100...
    http://diydrones.com/profiles/blogs/red-balloon-finder
     
  18. xtracrispy

    Joined:
    Sep 6, 2014
    Messages:
    7
    Likes Received:
    0
    I think that the sum total of a decent camera, powerful with gpu embedded computer, and wireless transmitter will be a few hundred dollars.
    Do you guys think a price somewhere between 500-1000 is reasonable, given the parts involved? That would place it about equivalent to the cost of a non-active gimbal.

    Cool! I looked into this as well. So it appears to do pure color tracking, which is computationally easier.

    One problem with this however, is that lighting conditions can change, the color signature of an object can get lost.
    Last but not least, it'd require the object to have a set color, which can lead to problems if the background also has the same color. (Imagine trying to track a green tshirt in a park!)

    This in part can be solved by "color coding", which tracks a sequence of colors to reject outliers.
    Here's another experiment I did with a system that costs $60 combined!
    https://www.youtube.com/watch?v=8SbWyUAWq9M&list=UUy6SYEB_Gk40j9NOy8RI5aQ
     
  19. Yaw Master

    Joined:
    May 29, 2014
    Messages:
    37
    Likes Received:
    3
    Plus one. This is precisely what we need. Personally, I had expectations that DJI would have inconcluded this in their automated waypoint upgrade. Go for it, please.
     
  20. DBeard

    Joined:
    Aug 13, 2014
    Messages:
    137
    Likes Received:
    1
    What camera are you using in that example youtube?