[TOOL][WIN] Offline TXT FlightRecord to CSV Converter

[isicom]

Why does everyone want to develop the same tool again and again? Can't we collectively work and make one tool strong and meaningful for all?

I'm d'accrod with you, alokbhargava. You all have right. We can not always ask for the code. If the sourcecode become public DJI starts again with re-encryting.

But I think, what manju and me could help is a working/executable version through a console call and compatible with linux in same time...

I not interesting for the code any more. But a non .exe file or a java file, working by console or a console python from creator of this tool would be enough, so we can run it on ubuntu console based OS. Maybe, he has time to build something for us

thanks

 

[manju91080]

Why does everyone want to develop the same tool again and again? Can't we collectively work and make one tool strong and meaningful for all?

Its not same tool again and again. I am not at all interested in windows executable. Tell me how this tool is really useful to those who are having different Operating system?
I am trying to understand how the log is encoded in the binary file so that it will be easy for me to read and convert it to meaningful content.

 

[manju91080]

I'm d'accrod with you, alokbhargava. You all have right. We can not always ask for the code. If the sourcecode become public DJI starts again with re-encryting.

But I think, what manju and me could help is a working/executable version through a console call and compatible with linux in same time...

I not interesting for the code any more. But a non .exe file or a java file, working by console or a console python from creator of this tool would be enough, so we can run it on ubuntu console based OS. Maybe, he has time to build something for us

thanks

I totally agree with you.

 

[alokbhargava]

Its not same tool again and again. I am not at all interested in windows executable. Tell me how this tool is really useful to those who are having different Operating system?
I am trying to understand how the log is encoded in the binary file so that it will be easy for me to read and convert it to meaningful content.

Unfortunately decoding algorithm is not published and many have experimented to decode on their own and with cooperation of others. We can't help if they don't let it be available publicly due to the reasons best known to them.

I believe friendship is the key to the success here.

 

[alokbhargava]

I'm d'accrod with you, alokbhargava. You all have right. We can not always ask for the code. If the sourcecode become public DJI starts again with re-encryting.

But I think, what manju and me could help is a working/executable version through a console call and compatible with linux in same time...

I not interesting for the code any more. But a non .exe file or a java file, working by console or a console python from creator of this tool would be enough, so we can run it on ubuntu console based OS. Maybe, he has time to build something for us

thanks

Have you ever seen encrypted logs? Why does DJI do it? Logs should be available to all for analysis. Now the question comes: what was the purpose of these logs? I believe it was to help DJI to analyze and understand the mishaps possibly to adjust the claims and improve the products.

You and I do not have access to basic encryption algorithms. Some have managed to decode it.

Msinger has made a web based system which is available to all users and is easy to use it. It may not give you the results in the format you wish to get it but it's a good starting point. Forward your suggestions to him to include more desired features.

Programming world is very competitive, every good programmer thinks he/ she can produce much better results. But there would be wonders if all like minded programmers sit together and work out the best of the best solution with cooperative efforts.

These are my two cents.

 

[manju91080]

Have you ever seen encrypted logs? Why does DJI do it? Logs should be available to all for analysis. Now the question comes: what was the purpose of these logs? I believe it was to help DJI to analyze and understand the mishaps possibly to adjust the claims and improve the products.

You and I do not have access to basic encryption algorithms. Some have managed to decode it.

Msinger has made a web based system which is available to all users and is easy to use it. It may not give you the results in the format you wish to get it but it's a good starting point. Forward your suggestions to him to include more desired features.

Programming world is very competitive, every good programmer thinks he/ she can produce much better results. But there would be wonders if all like minded programmers sit together and work out the best of the best solution with cooperative efforts.

These are my two cents.

I am a software developer with 12 years of experience. I started looking in to this recently and thought of looking for a help. Lets catch up offline. ( I am also staying in california)

 

[Digdat0]

I am a software developer with 12 years of experience. I started looking in to this recently and thought of looking for a help. Lets catch up offline. ( I am also staying in california)

i would encourage creating a brand new thread to find like minded folk who want to collaborate on a project. start a github area and start to pull in resources. there are some python and js code out there now if you hunt.

 

[skylow]

Why I do not find my battery printed SN on the output file of TXTlogToCSVtool ? It is strange because when I open the raw data file (encrypted) with the Blocnote, I can find the information in clear (not encrypted) : the battery SN is 0DQADBN03104X0

It seams the tool doesn't find the Battery SN information ?

 

[hunterxero]

Works for data coming out of my Phantom 4, which quad are you using? Also would be helpful to know the device (iphone etc.) and version of DJI GO app in order to troubleshoot.

 

[skylow]

It's a P4+

Here is the file. When I import it on airdata.com, I can find the battery SN correctly. Is it a bug in the tool ?

 

[hunterxero]

I took a look at this in Airdata and through the tool.

I think there is a device specific issue with TXTlogToCSVtool

Your battery serial number is not in the converted CSV file. I think this is because you're using an iOS device. In previous flights with my P4 and iPhone 7 plus, I've had corruption of the Battery Serial number

With my Android Nexus 6, the battery serial number converts fine.

The fields you want to check in the CSV are:

RECOVER.batterySn
CENTER_BATTERY.serialNo

As you noted, these fields are blank in the converted file, but are showing up in Airdata.

@ferraript, any known issues with iOS version of DJI GO (version 4.0.8) that might result in empty battery serial numbers?

 

[ferraript]

it's not a bug
your TXT log really has empty fields in RECOVER.batterySn and CENTER_BATTERY.serialNo
nonencrypted SN you found is in DETAILS section and the tool doesn't write some of those fields to csv

I can't tell anything about iOS versions except they are different from Android versions

 

[AJLebeau]

Hi folks, I'm glad to present you the result of my work.
A lot of us have been unhappy, because until now, there was no offline TXT FlightRecord to CSV Converter.
And after websites such as HealthyDrones and djilogs.com started to charge for their services, I decided I really need some offline converter. So I created it by myself.

It's quite simple, all you need to do is select input TXT file and output CSV file and press the GO button. There is also option to export JPG images, if they are present in the log.

I don't plan to add graphs, statistics or something like that.
My thanks goes to (for me) unknown person, who published some sources at this site, so I was able to put missing pieces together.

If you find any error, let me know.
Enjoy

Notice 1: It is confirmed that the TXT logs of Inspire, and also TXT logs from iOS app, have a little bit different structure (not known to me), so, please, don't be surprised when you see some strange values

Notice 2: Some of you requested the structure of the TXT log, so, here you have it too
- edit: since DJI GO 2.8.4, TXT logs are encrypted, so the structure file is useless

Notice 3: Tool can be also run from command line like this, example:
TXTlogToCSVtool "C:\temp\inputFile.txt" "C:\temp\outputFile.csv" J I
It will try to read inputFile.txt and write the output into outputFile.csv.
Optional parameters:
J ... it will also export JPGs, if they are present in the log
I ... imperial units will be used

Notice 4: The tool is freeware and it is intended for your own personal use only.

actual version is from 2017/01/13

Hello, and thanks for making this conversion tool. My issues are that no one seems to know or bother to tell anyone how to import this into dashware, and what profiles to be used (i.e. Aeris CompuMask, AR Drone, or any of the 50+ data profile choices it provides). Am I in the wrong thread here, or am I trying to do the wrong thing with the wrong program? I'll admit I'm blind right now. I thought that the .txt files could be converted to .CSV files which would provide the parameters to perform certain gauge layouts on top of the videos taken from the DJI Phantom 3 Pro. Can ANYONE direct me to the right thread if I am off course, and if not, could somebody help a poor old blind soul figure this out? Why must they make it SO difficult? :-{

 

[AJLebeau]

Hello, and thanks for making this conversion tool. My issues are that no one seems to know or bother to tell anyone how to import this into dashware, and what profiles to be used (i.e. Aeris CompuMask, AR Drone, or any of the 50+ data profile choices it provides). Am I in the wrong thread here, or am I trying to do the wrong thing with the wrong program? I'll admit I'm blind right now. I thought that the .txt files could be converted to .CSV files which would provide the parameters to perform certain gauge layouts on top of the videos taken from the DJI Phantom 3 Pro. Can ANYONE direct me to the right thread if I am off course, and if not, could somebody help a poor old blind soul figure this out? Why must they make it SO difficult? :-{

Hello, and thanks for making this conversion tool. My issues are that no one seems to know or bother to tell anyone how to import this into dashware, and what profiles to be used (i.e. Aeris CompuMask, AR Drone, or any of the 50+ data profile choices it provides). Am I in the wrong thread here, or am I trying to do the wrong thing with the wrong program? I'll admit I'm blind right now. I thought that the .txt files could be converted to .CSV files which would provide the parameters to perform certain gauge layouts on top of the videos taken from the DJI Phantom 3 Pro. Can ANYONE direct me to the right thread if I am off course, and if not, could somebody help a poor old blind soul figure this out? Why must they make it SO difficult? :-{

BTW, I actually had SOME success using DatCon with my .dat files, but they ALSO don't give the appropriate details on how to use their current "Dashware Templates" they have for download, and seemingly think that the only thing anyone is interested in is the battery amperage and motor revs. I had to delete a "Background" of some sort that was covering the video import and delete more than half the gauges on their templates JUST to get a simple video with a few overlays that I couldn't even choose as the telemetry apparently wasn't included or the appropriate field was not checked off in the "Analysis By Time" area. Can ANYONE put together a comprehensive way to use these .CSV fields as they relate to the actual gauge info, what's required, etcetera? A LOT OF WORK has already been done, but to assume that now the hard part has been figured out doesn't mean the rest of us have a clue here.............LOL
I

 

[BudWalker]

BTW, I actually had SOME success using DatCon with my .dat files, but they ALSO don't give the appropriate details on how to use their current "Dashware Templates" they have for download, and seemingly think that the only thing anyone is interested in is the battery amperage and motor revs. I had to delete a "Background" of some sort that was covering the video import and delete more than half the gauges on their templates JUST to get a simple video with a few overlays that I couldn't even choose as the telemetry apparently wasn't included or the appropriate field was not checked off in the "Analysis By Time" area. Can ANYONE put together a comprehensive way to use these .CSV fields as they relate to the actual gauge info, what's required, etcetera? A LOT OF WORK has already been done, but to assume that now the hard part has been figured out doesn't mean the rest of us have a clue here.............LOL
I

Did you see this
DatCon Dahware Profiles

 

[AJLebeau]

Does ANYONE know of a demo video that shows HOW to import a converted .txt to .CSV file into "Dashware" so that we can import BASIC data like heading, speed, distance traveled, etc? I have seen the videos out there, but none of them bother explaining HOW to go about creating such. PLEASE SHARE YOUR DISCOVERIES with the rest of the flock!! PLEASE!!

 

[BudWalker]

Hello, and thanks for making this conversion tool. My issues are that no one seems to know or bother to tell anyone how to import this into dashware, and what profiles to be used (i.e. Aeris CompuMask, AR Drone, or any of the 50+ data profile choices it provides). Am I in the wrong thread here, or am I trying to do the wrong thing with the wrong program? I'll admit I'm blind right now. I thought that the .txt files could be converted to .CSV files which would provide the parameters to perform certain gauge layouts on top of the videos taken from the DJI Phantom 3 Pro. Can ANYONE direct me to the right thread if I am off course, and if not, could somebody help a poor old blind soul figure this out? Why must they make it SO difficult? :-{

Check out post #215 in this thread.

 

[AJLebeau]

Hi folks, I'm glad to present you the result of my work.
A lot of us have been unhappy, because until now, there was no offline TXT FlightRecord to CSV Converter.
And after websites such as HealthyDrones and djilogs.com started to charge for their services, I decided I really need some offline converter. So I created it by myself.

It's quite simple, all you need to do is select input TXT file and output CSV file and press the GO button. There is also option to export JPG images, if they are present in the log.

I don't plan to add graphs, statistics or something like that.
My thanks goes to (for me) unknown person, who published some sources at this site, so I was able to put missing pieces together.

If you find any error, let me know.
Enjoy

Notice 1: It is confirmed that the TXT logs of Inspire, and also TXT logs from iOS app, have a little bit different structure (not known to me), so, please, don't be surprised when you see some strange values

Notice 2: Some of you requested the structure of the TXT log, so, here you have it too
- edit: since DJI GO 2.8.4, TXT logs are encrypted, so the structure file is useless

Notice 3: Tool can be also run from command line like this, example:
TXTlogToCSVtool "C:\temp\inputFile.txt" "C:\temp\outputFile.csv" J I
It will try to read inputFile.txt and write the output into outputFile.csv.
Optional parameters:
J ... it will also export JPGs, if they are present in the log
I ... imperial units will be used

Notice 4: The tool is freeware and it is intended for your own personal use only.

actual version is from 2017/01/13

Is this to be used as a way to import datasets into Dashware for Video Overlays? That's what I was looking for, but your conversion tool doesn't say anything about how to import it, what type of profile to use, or any rather pertinent data. Am I "Barking up the wrong Tree" here?

 

[AJLebeau]

Check out post #215 in this thread.

Hello BudWalker,

Thanks for your timely and diligent response; I REALLY appreciate your assistance with this.

I did indeed find the post, and download the mentioned "txt to csv converter" utilities. I do have one question I hope you can help me with without having to scour the entire contents of this download. As it does not seem to contain an executable file, is it safe to assume this is contains only a TEMPLATE for use with Dashware creations? If this is indeed the case (as my cursory preview of the readme file seems to elude to) what utility is/has been used to make the conversion from TXT to CSV with use for this?

I have downloaded a FREE "txt to csv converter" executeable from thread #1 of the following URL: [TOOL][WIN] Offline TXT FlightRecord to CSV Converter

Would that be related or do you know if it's compatible in it's conversion? If not, could you recommend an appropriate conversion tool that is FREE and OFFLINE?

Many of the "Free" sites in these older threads have all gone to a "Pay Format" or some sort of collaborative consortium that promises "unbelievable options" but you must pay, upload your .TXT file, and in most cases do not get to download and keep your .CSV file conversion.

On a PERSONAL NOTE and of no reflection towards you Mr. BudWalker,

"While I have found that most everyone in this forum has been VERY Generous with their time, effort, and advice (including yourself) there are others that seemingly feel as though they need guard their discoveries or be compensated for their efforts with special reference to THIS PARTICULAR ISSUE. To those of you who feel the need to benefit from your efforts along the course of THIS and any OTHER issues that are discussed in this forum, I would ask that you kindly find another venue with which to "Advertise" your product and quit wasting everyone's time, as this venue, as I have been told, (and is listed in the rules of conduct), that no such product advertising or promotion activity should occur. One can "Refer" to a product that performs a function or a equipment piece that fulfills a need, but for the base of this venue most Pilots are looking for the "Good Natured Advice and the Sharing of Information" to HELP their fellow pilots, not GOUGE their pockets.

Again, thank you sincerely BudWalker, and all you others out there who are sincere and have done an unselfish service to countless pilots (and wanna-be's Like Me) a true service with our experiences.

Best Regards Sir,


A.J. Lebeau
Computer Network Systems Administrator/Engineer, (Ret.)
M.C.S.E., M.C.P. + I., M.C.P., CompTIA+
[email protected]

 

[AJLebeau]

Check out post #215 in this thread.

Dear Mr. BudWalker,
Please disregard; I am not familiar with how to navigate these boards/threads/posts very well, not being one for social media. I DID ascertain that the conversion tool I downloaded INDEED appears to be the correct TXT to CSV Conversion tool, am I correct? (I decided to jump back to #1 in the post, and sure enough, there was the URL I referred you to, prior to post #215).

Again, THANK YOU for your assistance!


A.J. Lebeau
Computer Network Systems Administrator/Engineer, (Ret.)
M.C.S.E., M.C.P. + I., M.C.P., CompTIA+
[email protected]