Welcome to PhantomPilots.com

Sign up for a weekly email of the latest drone news & information

[TOOL][WIN] Offline TXT FlightRecord to CSV Converter

Discussion in 'Pro/Adv Discussion' started by ferraript, Mar 6, 2016.

  1. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    228
    Likes Received:
    75
    Location:
    Slovakia/Kosice
    Hi folks, I'm glad to present you the result of my work.
    A lot of us have been unhappy, because until now, there was no offline TXT FlightRecord to CSV Converter.
    And after websites such as HealthyDrones and djilogs.com started to charge for their services, I decided I really need some offline converter. So I created it by myself.

    It's quite simple, all you need to do is select input TXT file and output CSV file and press the GO button. There is also option to export JPG images, if they are present in the log.

    I don't plan to add graphs, statistics or something like that.
    My thanks goes to (for me) unknown person, who published some sources at this site, so I was able to put missing pieces together.

    If you find any error, let me know.
    Enjoy ;)

    Notice 1: It is confirmed that the TXT logs of Inspire, and also TXT logs from iOS app, have a little bit different structure (not known to me), so, please, don't be surprised when you see some strange values

    Notice 2: Some of you requested the structure of the TXT log, so, here you have it too
    - edit: since DJI GO 2.8.4, TXT logs are encrypted, so the structure file is useless

    Notice 3: Tool can be also run from command line like this, example:
    TXTlogToCSVtool "C:\temp\inputFile.txt" "C:\temp\outputFile.csv" J I
    It will try to read inputFile.txt and write the output into outputFile.csv.
    Optional parameters:
    J ... it will also export JPGs, if they are present in the log
    I ... imperial units will be used

    actual version is from 2016/11/27
     

    Attached Files:

    #1 ferraript, Mar 6, 2016
    Last edited: Nov 27, 2016
  2. Quadcopterpilot

    Joined:
    Nov 25, 2015
    Messages:
    260
    Likes Received:
    69
    Location:
    Grimsby, Ontario, Canada
    Thanks for doing this, was hoping I didn't have to do it myself.
    However I'm getting errors each time I use it on different files and not getting any usable data.
    Error in 0f/firmware at byte 12
    Error in 01/OSD at byte 123
     
  3. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    228
    Likes Received:
    75
    Location:
    Slovakia/Kosice
    hm, maybe logs from another types of Phantoms have another structure
    I have P3 Advanced
    What type do you have?
    Could you upload at least one of those TXT logs somewhere? (if they are private for you, send me PM)
     
  4. Quadcopterpilot

    Joined:
    Nov 25, 2015
    Messages:
    260
    Likes Received:
    69
    Location:
    Grimsby, Ontario, Canada
    I have P3P, will try to get you a file
     
  5. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    228
    Likes Received:
    75
    Location:
    Slovakia/Kosice
    tool updated
    it can now deal with records of various size and also images embedded between the records
     
  6. Quadcopterpilot

    Joined:
    Nov 25, 2015
    Messages:
    260
    Likes Received:
    69
    Location:
    Grimsby, Ontario, Canada
    Sorry, still having some issues:
    Program can't start because qtintf.dll is missing
     
  7. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    228
    Likes Received:
    75
    Location:
    Slovakia/Kosice
    @Luap provided me with some logs, so hopefully this new version will work fine

    changes:
    added possibility to choose CSV delimiter
    variable RC-record size enabled
    qtintf.dll no longer needed
     
  8. Quadcopterpilot

    Joined:
    Nov 25, 2015
    Messages:
    260
    Likes Received:
    69
    Location:
    Grimsby, Ontario, Canada
    Excellent, works great, thanks!!
    Will do some data mining and let you know how I do...Thanks Again for this!!
     
  9. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,236
    Likes Received:
    915
    Location:
    Indy, USA
    There is some differences between android and ios OS's. But I don't think they matter in this case due to the limited amount of fields.

    What you've put together here is very nice. I appreciate your efforts big time! This allows the txt file to be converted anywhere anytime even when there's no Wifi connection.

    Currently there is a few minor issues but more importantly, nothing major. Do you have any plans to add any more fields? What are the chances of doing a setup similar to the "comprehensive.csv" on dji.com that has roughly 150 fields?
     
    #9 flyNfrank, Mar 9, 2016
    Last edited: Mar 9, 2016
  10. Trackman1

    Joined:
    Jan 1, 2016
    Messages:
    839
    Likes Received:
    212
    Location:
    Ontario, Canada
    Worked perfectly. Thanks a bunch.
     
  11. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    228
    Likes Received:
    75
    Location:
    Slovakia/Kosice
    last time I used their converter, I got 170 columns
    my tool gives 174 columns
    I think that's more than enough :)

    at the moment I am looking into decompiled sources of DJI GO, but it's all mixed up
    I don't know if I can get more data
     
  12. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,236
    Likes Received:
    915
    Location:
    Indy, USA
    I Happily stand corrected! I must have converted the file and then opened the wrong one to reference. I just quickly and notice there was much more columns then I saw yesterday. I will look them over and see what we have to work with. Will you need a dashware profile created? If so I can send you a convo.
     
  13. Quadcopterpilot

    Joined:
    Nov 25, 2015
    Messages:
    260
    Likes Received:
    69
    Location:
    Grimsby, Ontario, Canada
    I get more data than I can handle.
    Busy hiding fields I'm not interested in or that have no data.
     
  14. flyNfrank

    Joined:
    Feb 1, 2014
    Messages:
    4,236
    Likes Received:
    915
    Location:
    Indy, USA
    @ferraript
    Surely you're not complaining about something that is free.

    I know for a fact this @ferraript put in some serious hours and effort to put this together. I know I sure as heck appreciate what has been put together.
     
  15. Quadcopterpilot

    Joined:
    Nov 25, 2015
    Messages:
    260
    Likes Received:
    69
    Location:
    Grimsby, Ontario, Canada
    Not at all, this is great.
    Was just commenting as someone mentioned they didn't seem to get much data
     
    BudWalker likes this.
  16. msinger

    Approved Vendor

    Joined:
    Oct 30, 2014
    Messages:
    19,103
    Likes Received:
    5,627
    Location:
    US
    flyNfrank, did you do any comparison between this tool and the $1 converter to see how the aileron, elevator, throttle, rudder, and gimbal match up? Something seems off, but I'm not sure which is incorrect.
     
  17. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    228
    Likes Received:
    75
    Location:
    Slovakia/Kosice
    these values should be fine
    I just changed the scale, so it is the same like @BudWalker is using in his DAT converter, from -10000 to 10000
     
  18. BudWalker

    Joined:
    Jun 24, 2015
    Messages:
    782
    Likes Received:
    231
    Location:
    Santa Margarita, Ca
    The .txt file produced by the Go App uses a [0, 2048] range. I suspect this is the output of the control stick digitizer. Somewhere between that output and the .DAT field that range gets changed to [-10000, 10000]. I.e. DatCon doesn't change the scale, it just reports what it sees in that field.
    @msinger
     
  19. ferraript

    Joined:
    Feb 3, 2016
    Messages:
    228
    Likes Received:
    75
    Location:
    Slovakia/Kosice
    I have to correct you, it uses range 0x016C ( = 364) .. 0x0694 ( = 1684)
     
  20. BudWalker

    Joined:
    Jun 24, 2015
    Messages:
    782
    Likes Received:
    231
    Location:
    Santa Margarita, Ca
    It's been a while since I was looking at .txt files. I seem to remember that the value for the neutral position is 1024. Did I get that right? I must've just assumed that 1024 being neutral means that the interval is [0, 2048]