Welcome to PhantomPilots.com

Sign up for a weekly email of the latest drone news & information

Percepto-supercomputer technology revolutionizing drones

Discussion in 'General Discussion' started by ravivraz, Feb 4, 2015.

  1. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    Hi,
    As promised, more details are now available. NVIDIA is our technological partner and Percepto is powered by the Tegra K1 SoC.
    Thanks a lot for your feedback and interest in Percepto.
    After getting many repeated questions I have decided to answer some of them here to clarify some things.


    What exactly is Percepto?
    Percepto is a drone add-on. Meaning it’s a device (about the size of a deck of cards) and a small camera that you connect to your existing drone. Percepto enables new features you didn’t have before. It enables your drone to use the power of computer vision in real time. With computer vision drones can “see” and understand the environment better than with todays sensors.
    What can I do with it?
    We will ship Percepto with a few applications, like:
    Our filming app (Name ideas are welcomed)
    Using your smartphone, mark any object and have the drone follow it, circle it or create any filming maneuver you desire with a click of a button. Since Percepto works onboard and in high rates it is very responsive and can create filming maneuvers in higher speeds and smoothness than todays GPS based features. We are constantly working with professional aerial photographers to improve Percepto’s filming capabilities.There is also a mode where Percepto controls the gimbal alone, leaving the drone control in your hands.
    OSD (On Screen Dispaly) app (Again name ideas are welcomed)
    A customizable heads up display, driven by our computer vision algorithms which can produce smart alerts such as obstacle advisory, object searching or even locations of additional Percepto powered drones flying nearby (dogfights, anyone?).
    Percepto will constantly update with applications developed by us, as well as applications developed by our community. So, if you’ll own a Percepto, you will be able to experience any new exciting application released.
    What else can I do with it?
    You can develop and use your own computer vision based applications for your drone. By using our drone control API and pre-accelerated computer vision primitives, you can focus on your application, and get results a lot faster.
    For example, let’s say you want to control your drone using hand gestures. With Percepto, you already got the hardware ready, so you can either write your own code or use Percepto’s core hand recognition primitive, as well as our APIs to easily direct the drone’s movement. Now all you need to do is decide what the drone should do after each gesture, and write a simple code that describes just that. After connecting Percepto, you’ll have a hand gestured controlled drone.
    Percepto is open source making it a great platform for sharing your new application with other drone users.
    Does Percepto include a flight controller?
    No, it integrates with existing flight controllers like 3DR Pixhawk and DJI Naza. After considering safety issues, ease of development, processing resources and the commercial drones ecosystem benefits, we came to the conclusion that the computer vision unit should be set apart from the flight controller.
    How do I physically connect it to my drone?
    Percepto connects to your drone the same way a camera gimbal, an FPV transmitter or DJI Lightbridge connects to your drone. It needs power from the drone battery, and a connection to the drone’s flight controller. All cables will be included.
    Which drones can I connect it to?
    We’ve already tested Percepto with 3DR Pixhawk and DJI Naza controllers, so, every drone that uses one of these controllers is compatible, but basically, you can connect it to every drone that has a PPM input in its flight controller. Also, the drone can’t be too small, as it has to carry Percepto, which will weigh around 5 ounces (140 grams).
    How is Percepto different from other development platforms?
    Percepto is a state of the art processing platform tailored specifically for drones. The connectivity is designed for this specific use. The camera we’re shipping with Percepto is focused on frame rate and minimal distortion. Above all, our computer vision primitives are accelerated on Percepto’s CPU/GPU (NVIDIA Tegra K1) and our APIs make it easy to translate software decisions to drone flight maneuvers.
    How will Percepto control the drone?
    Depending on the application and the flight controller in use, Percepto can control the drone through PPM (just like an R/C receiver), or it can communicate with the flight controller through a digital connection (MAVLink, for example). Percepto can also be set to control only part of the control channels, leaving the others for manual control.
    Percepto itself can be controlled either by your RC remote, or with your smart device through WiFi.
    For some features, “follow me” for example, Percepto might fly the drone completely autonomously. In other cases, such as obstacle avoidance, Percepto might only intervene partly with the drones flight route or manual control.
    Another option is to use Percepto for completely different tasks like augmented reality FPV or performance monitoring.

    When will it be available?
    We will start shipping Percepto to selected beta testers in July 2015. It will become publicly available by September.
    How much will it cost?
    We are now finalizing mass production costs with our different suppliers. This is why we have decided to launch our product in a crowdfunding campaign soon. I know it’s already became a cliche but it’s true, the more products we will have pre-booked and the more demand we will see for the product the lower we can make the cost.

    For registration and more information go to www.Percepto.co

    Raviv
     

    Attached Files:

  2. Khudson7

    Khudson7 Guest

    Which programming language is your API using?
     
  3. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    Thanks for asking.
    It is using cpp.
     
  4. Khudson7

    Khudson7 Guest

    A few more questions. You mentioned the processor you are using inside, and you will have an API, which I am guessing will be reading data and allowing a programmer to write functionality based on that data. Where is this data coming from. For instance, the diagram shows sensors from the flight controller sending data to the processor. Am I right in assuming it will be reading things like altitude, speed, gps coordinates, battery levels, signal strengths , direction, etc. coming from the Flight Controller? Will there be additional sensors in your hardware(you mentioned the camera) like IR sensors, or sonic sensors, to provide additional data to do sense and avoidance maneuvering, or hand gesturing that you mentioned? Are there other sensors built into your hardware providing any other data with which a computer program can operate on?

    Am I right in assuming you hardware via the api will then be able to send back signals/instructions to the flight controller to do things like maneuvering and/or controlling the gimbal? Is this the basic idea behind this?

    I apologize if these sound a bit vague or basic to you, just trying to get a better understanding of the details as to how this will work. If I am touching on areas that you are not ready to reveal yet, just say so, I understand, it is still early.

    EDIT: I really do like this concept of adding a computer(processor) onboard. It opens up a world of potential abilities over what a drone is able to do today, IMHO. In no way are my questions above, meant as a critique, just trying to understand where you are going with this, and what other hardware might be included or might be an add-on down the road(if not included), to be able to do some of the things you are alluding to.
     
  5. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    @ khudson7 Your assumption is correct, it depends on the availability of the data by the flight controller. Percepto includes its own camera like mentioned, you may add additional cameras to it. Percepto delivers vision as a sensor.
    The purpose of the API is to allow reuse of the core code we are developing here in Percepto. It will include procedures for drone control as well as computer vision primitives and connectivity to the ground app.
    The general idea is using all the data Percepto is processing to control the drone and gimbal. Percepto computing power also allows more advanced AI.
    Hope to have answered all your questions. Feel free to ask more, it is good for us and the followers to understand.

    Raviv
     
  6. PhantomFanatic

    Joined:
    Nov 14, 2014
    Messages:
    1,143
    Likes Received:
    5
    Location:
    USA, GA
    Unless I missed something, it has an onboard camera, but you say it can be mounted anywhere. It would seem the camera should be foward facing, but with a 3D gimbal and a GoPro, where could it possibly go? Or, does it face downwards??

    Sounds interesting. :)
     
  7. Khudson7

    Khudson7 Guest

    Ok, got it, thanks. So using both data from the flight controller, plus other algorithms you have developed along with using info from camera recognition as one source and other AI functionality. Sounds really interesting. You mentioned using C++ and open source. Does that mean all of the code and algorithms you have developed internally, will also be available to open source programmers, to use and modify? Or are the internals of the API coding, proprietary?
     
  8. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    @ PhantomFanatic - it can be easily mounted on the gopro it self (see our facebook page for more images)
    @ Khudson7 - completely open source.
     
  9. ianwood

    ianwood Taco Wrangler
    Staff Member

    Joined:
    Jan 7, 2014
    Messages:
    4,910
    Likes Received:
    1,788
    Location:
    Lost Angeles
    Amps under load? Weight? Any thoughts towards linking this with a ground controller? Provide a ground based app into it and you can unlock a lot of industrial applications (once the FAA pulls their head out of their backside).

    Pedantic mode: what is the "supercomputer" technology? Kind of an archaic reference.
     
  10. gbello23

    Joined:
    Jan 4, 2015
    Messages:
    51
    Likes Received:
    0
    Location:
    Toronto, Ontario
    Have you looked into vision/vision+ application?
     
  11. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    @ ianwood - At max load it stands at 15 Watts, if using a 3s battery it uses 1.35 Amps. On average it uses about 7 watts which is 0.46 Amps. Weight is less than 200 grams. We do provide a smart device app which can be used as a ground station. And about the “supercomputer technology”, point taken. NVIDIA’s Tegra k1 is just an amazing piece of technology that is all I meant….
    @ gbello23 - Percepto can control both vision and vision+. Our filming app gives amazing new features.
     
  12. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    Hi all,
    I posted here a while back regarding a product we are launching soon on Indiegogo called Percepto. Using Percepto’s camera to see its environment, “The director” app gives the drone a professional cameraman’s moves. Below are a few examples we thought of and roughly tested. We would love to get your ideas and thoughts.
    https://www.youtube.com/watch?v=zq6di3YjJFg
    more info: http://bit.ly/1BcsVmN
     
  13. wkf94025

    Joined:
    Jan 19, 2014
    Messages:
    608
    Likes Received:
    8
    Location:
    St. Helena, CA
    Concerned about weight. Any chance your vision apps could utilize GoPro?
     
  14. CarlJ

    Joined:
    Feb 24, 2014
    Messages:
    782
    Likes Received:
    19
    Location:
    Peru, Indiana
    You fools!!! Am I the only one here who watches TV? Is no one concerned about this thing becoming self-aware and going all robot wars up in this *****? Oh sure it'll say "I'll be back", but pretty soon the wait will be longer and longer, because it's out looking for "self improvement" which we all know is code for sleeping with my best friend, and cleaning out my bank accounts.

    But my question is: does this work with any model Phantom?
     
  15. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    Works with phantom 1 and 2
     
  16. ravivraz

    Joined:
    Dec 22, 2014
    Messages:
    36
    Likes Received:
    0
    Our indiegogo campaign is now live.
    Thank you for your comments, insights and suggestions.
     

    Attached Files: