Documenting the Format of DJI Log Files

DJI has not publicly documented the format of their log files. However, we have reverse engineered the format of DJI's ".txt" log files, which are recorded by the controller (usually in a smartphone app) during each flight. Despite the filename suffix ".txt", these are not text files. Instead, they are binary files (that contain very little 'text').

Format of the ".txt" log file

All multi-byte numeric values are little-endian.

The file is laid out as follows:

  1. A 100-byte header
         (This was smaller in older versions of the log file.)
  2. A records area
  3. A details area
The parsing of a ".txt" file is illustrated by the source code for the "djiparsetxt" application, described below.

Format of the header

  1. The first 8 bytes of the file give you the size of the header and the records area combined - i.e., 100 plus the size of the records area.
         (Or it might just be the first 4 bytes of the file. But the following 4 bytes always seem to be zero, and multi-byte words are stored in little-endian order, so the two descriptions are equivalent.)
  2. The next 4 bytes of the file appear to be a version number (for the log file format). The 3rd byte seems to be an indicator of significant changes in the log file format.
  3. The remaining 88 bytes of the header (not present in older versions of the log file) are all zero.

Format of the records area

The "records area" consists of a sequence of records. Each record consists of:
  1. A 1-byte record type
  2. A 1-byte record length
  3. "record length" bytes of record payload
  4. A 1-byte 'end of record' marker, with the value 0xFF.
        (This 'end of record' marker is redundant, because of the existing record length byte.)
        (JPEG records - described later - are somewhat different; they don't have the 0xFF 'end of record' marker, and the record length is ignored.)
The parsing of a record is illustrated by the source code for the "parseRecord()" function.

Format of the record payload

Payload scrambling
In recent versions of the log format, the contents of each record payload' have been obfuscated - i.e., scrambled, using an XOR (i.e., exclusive-OR)-based mechanism. It's not obvious why DJI has chosen to do this. Perhaps they are concerned about people falsifying the contents of these logs, and chose to scramble the record payloads to try to make this harder? (However, a much better way to prevent people modifying the contents of logs would have been to add a digital signature, leaving the payloads in 'plaintext'.)

The scrambling is done as follows:

The resulting unscrambled_payload (length: record length-1) is interpreted, depending on the record type, as follows:
  1. OSD
  2. HOME
  3. GIMBAL
  4. RC
  5. CUSTOM
  6. DEFORM
  7. CENTER_BATTERY
  8. SMART_BATTERY
  9. APP_TIP
  10. APP_WARN
  11. RC_GPS - format unknown
  12. RC_DEBUG - format unknown
  13. RECOVER
  14. APP_GPS
  15. FIRMWARE
  16. OFDM_DEBUG - format unknown
  17. VISION_GROUP - format unknown
  18. VISION_WARN - format unknown
  19. MC_PARAM - format unknown
  20. APP_OPERATION - format unknown
  21. (–23.) unknown
  22. APP_SER_WARN - format unknown
  23. (–56.) unknown
  24. JPEG - format described below
  25. (–254.) unknown
  26. not used
JPEG records are formatted differently from the others. They are not scrambled, and do not end with a 0xFF byte. Also, the record length byte, although present, is not used.

After two initial 'zero' bytes, a JPEG record consists of a sequence of (zero or more) JPEG images. Each JPEG image begins with a JPEG SOI ("start of image") marker: 0xFF 0xD8, and ends with a JPEG EOI ("end of image") marker: 0xFF 0xD9. An EOI marker that is not immediately followed by a SOI marker (beginning another JPEG image) marks the end of the JPEG record.

Our "djiparsetxt" application - described below - automatically extracts (into ".jpg" files) any JPEG images that it finds within JPEG records.

Format of the details area

The format of the details area (at the tail of the file, following the records area) is illustrated by the source code for the "parseDetails()" function.

Problems to be solved

If you can help resolve any of these, please let us know on our public mailing list, described below.
  1. We currently know only 0x1000 (i.e., 4096) rows of the scramble table. We need a table at least twice as large to unscramble all of the record types that we see. (Alternatively, there might be a way to generate the scramble bytes algorithmically, rather than having them stored in a table. Some people have suggested that this could be done using a CRC.)
  2. There are several record types that have been given a name, but which we do not currently know how their payloads are formatted. These include:
    1. (==0x0B) "RC_GPS"
    2. (==0x11) "VISION_GROUP"
    3. (==0x12) "VISION_WARN"
    4. (==0x13) "MC_PARAM"
    5. (==0x14) "APP_OPERATION"
    6. (==0x18) "APP_SER_WARN"
  3. We have seen other record types for which we do not know neither the name nor the format. These include:
    1. (==0x16)
    2. (==0x19)
    3. (==0x1a)
    4. (==0x1e)
    5. (==0x28)

The "djiparsetxt" application

"djiparsetxt" is a command-line application that reads a DJI '.txt' file and outputs - to stdout - the contents of the file in comma-separated value (i.e., "CSV") form. This output can then be used as input to a spreadsheet, or it could be piped into another application.

The application is provided as (C++) source code; not as a pre-built binary application. To use it, you must:

The application's source code is available - as the file "djiparsetxt-latest.tar.gz" - here. Use "tar -xf" to extract the code; then cd to the source code directory. Then run: make

Running the "djiparsetxt" application

The simplest way to run "djiparsetxt" is to use it to convert a DJI '.txt' log file into a CSV (i.e., comma-separated value) file that you can view in a spreadsheet:
    djiparsetxt 'input-txt-file-name' > output-csv-file-name
E.g., if your input log file is named "test.txt", you could run:
    djiparsetxt 'test.txt' > test.csv
In this particular case, the single-quote marks (') around the input file name aren't necessary. However, DJI log file names often contain square-bracket characters ([ and ]) that will need to be 'quoted' - e.g.
    djiparsetxt 'DJIFlightRecord_2017-09-01_[16-42-09].txt' > test.csv
Note that in addition to outputting a CSV file to 'stdout', the program also outputs lots of diagnostic information to 'stderr'. You can usually just ignore this.

If you wish to change the fields that are output to the CSV file (e.g., change the number of output fields, and/or their order), then you could do so by modifying the code for the outputOneRow() function.


The "generateKML" application

"generateKML" is a shell script - bundled along with the "djiparsetxt" software - that reads a CSV file (from 'stdin'), and converts it to a KML-format file that can be read by "Google Earth" to display the drone's track.

"generateKML" is written in the Tcl programming (scripting) language. To run it, you must have Tcl (including the "tclsh" shell) installed on your system; you can get this software here, or using your OS's package/port system.

Running the "generateKML" application

The simplest way to run "generateKML" is to use it to convert a CSV file (that you previously created by running "djiparsetxt") into a KML-format file (that you can then view using Google Earth) - e.g.
    generateKML < test.csv > test.kml

Alternatively you can avoid creating a file "test.csv", by piping the output from "djiparsetxt" directly to the input of "generateKML" - e.g.,

    djiparsetxt 'test.txt' | generateKML > test.kml

The "dji-log-discuss@lists.live555.com" mailing list

We have a public mailing list
  dji-log-discuss@lists.live555.com
for discussion of DJI's log file formats (and the "djiparsetxt" application). Note, however, that before you can post to the mailing list, you must first
subscribe to it. (This is standard for all Internet mailing lists; it helps protect against spam.)

(Note that you must subscribe to the mailing list using the same "From:" address that intend to use to later post messages to the list.)


Thanks

Many thanks to the authors of the following document and software, which were helpful for this reverse engineering:


Copyright Live Networks, Inc. All Rights Reserved