How to Extract a Device Tree File from Android Firmware Files

Up to now, all our cheap Android devices were based on older Linux kernel (3.0.x, 3.4.x) that still used board files (arch/arm/board, but we’ve recently seen companies like Amlogic and Rockchip release source code with Linux kernel 3.10.x. One of the key differences between these version are the move from board files to flattened device tree and multi-platform support. If it is fully implemented, a single kernel image should be able to boot multiple hardware platforms, and all low level configuration handled by the device tree file. Since I’ve connected the serial port of Tronsmart Vega S89 for debugging, and it’s a slow news day, I thought I might try to boot the Linux kernel I compiled myself, but one of the challenge was to get the device tree file. I’ll show how to extract it from the firmware. It should also be possible to get it directly from the flash, but “cat /proc/mtd” does not show a complete list of partition as in previous versions.

I’ve performed the steps below in Ubuntu 14.04. The first thing is to install some tools: the device tree compiler that we’ll use to decompile the dtb (binary) file into a dtd (text) file, and split_bootimg.pl a standard PERL script to extract files from boot.img:


I’ll use M8 / TM8 firmware (Amlogic S802) as an example. The exact procedure will vary between firmware files, but if you can boot.img, the procedure should be platform independent and work for any ARM SoC. After having downloaded and extracted the firmware file (TM8 ap6330_03102014A_0410_ROOT.rar), let’s create a working directory, and unzip the “OTA” file.


We now get a bunch of files, including boot.img. Great! Time to run split_bootimg.pl script to extract its content:


So we’ve got the kernel, a ramdisk, and a “second file” that happens to be the dtb file. We can now decompile it with dtc (device tree compiler) as follows:


That’s it. Here’s M8 device tree file.

I’ve done the same for Tronsmart Vega S89 (Elite). S89 firmware is usually distributed as an IMG file to be used with AML Flash Burning tool, but I haven’t found a way to extract such file yet. however, I’ve found an “OTA” firmware, to be updated via SD Card, on freaktab, and could extract the device tree file for Tronsmart Vega S89 Elite & Vega S89. Both M8 and S89 Elite DTD files are very similar, but the maximum CPU frequency seems to be higher in M8, and there are other apparently minor differences. Vega S89 DTD file appears to be much different however.

Share this:
FacebookTwitterHacker NewsSlashdotRedditLinkedInPinterestFlipboardMeWeLineEmailShare

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress

ROCK 5 ITX RK3588 mini-ITX motherboard

57 Replies to “How to Extract a Device Tree File from Android Firmware Files”

  1. @m][sko
    I’m loading the built boot.img via tftp and running it from there.
    Before it could boot to command line (in the ramdisk), but no Ethernet.
    With the right device tree file, I get Ethernet, and access to the command line. HDMI output seems to work but it’s all black. I was expecting the Android UI to show up as well. It’s probably because I need to modify the ramdisk. There are some pinmux conflict in the log as well, because of a camera module which is not present in the stock firmware. Expect progress to be slow, as I play on this only if I can’t find much to write about during the day.

    @Alain Theriault
    If you want to give it a try, here’s what I’m planning to do:
    1. Get console access to your M8 – Similar to http://www.cnx-software.com/2014/05/07/how-to-open-tronsmart-vega-s89-elite-and-access-the-serial-console/
    2. Build the kernel with the M8 DTD file (modify mk_m8.sh)- http://www.cnx-software.com/2014/03/10/amlogic-gpl-source-code-release-kernel-3-10-u-boot-and-drivers-wi-fi-nand-tvin-mali-gpu/ + post above
    3. Configure a tftp and nfs server in your Linux PC
    4. Copy to your boot.img to the tftp directory, and an ARM Linux rootfs to the NFS share (e.g. ALIP: http://releases.linaro.org/13.11/openembedded/vexpress-lsk/linaro-image-alip-genericarmv7a-20131126-163.rootfs.tar.gz)
    5. Try to boot with an adaptation of the following: http://forum.xbmc.org/showthread.php?tid=152328&pid=1304803#pid1304803

    If you want to make sure the kernel is working properly, you may want to boot Android instead. As mentioned to misko answer, your may need to modify the ramdisk, or just take the one from M8 firmware.

  2. hey cnxsoft, don’t know if it’s of any use – but I extracted the partitions with boot.img etc. on my Beelink 16GB (Tronsmart Standard S89 hardware-a-like) if those files are of any use to putter around in?

    I’m mostly a windows guy, I need to set up a *nix box for playing around on (mm.. maybe I could use my raspberry pi for that).

  3. M8 is better then S89 Elite?
    Antutu benchmark? I Can’t find anything good about m8 (S802).

    Thanks

  4. @Dante
    Both are about the same. But when I tried M8 tended to hang from time to time. It could have been hardware (overheat) or firmware issue. Not sure.

    @gizmomelb
    Did you get the boot.img from the NAND Flash or firmware file? If from the NAND flash, how did you do? I can’t find the partitions in my Vega S89 Elite.
    If you are a Windows guy, and want to play around, you’d better install VirtualBox + Ubuntu on your windows PC, especially if you’re going to give a try at building the kernel. The Raspberry Pi use would be limited (tftp / nfs server), hence unnecessary.

  5. hey cnxsoft – I extracted the boot.img from the NAND using adb shell and some help from Finless Bob on freaktab.

    http://www.freaktab.com/showthread.php?12472-factory-default-firmware-images&highlight=

    I have the extracted files from the 100k4 NAND, I’ve since updated to 101k4 but haven’t extracted the files again (is there any point if we can unpack the firmware update?).

    I’m happy to help out in any way I can. I have two Beelink M8 round boxes, one for testing – my USB -> serial cable/adapter arrived today as I need to update my Gotek floppy emulator (another project).

  6. @Dante

    Hi Dante! The square S802 boxes are going by the M8 brand, however Beelink have a round (Tronsmart S89 clone essentially – same motherboard, different firmware but you can flash with the S89 firmware) S802 device which is named M8 (and another one named S82). Confusing!

  7. @cnxsoft

    Hi cnxsoft – yeah I’ll go virtualbox or if I look around I can probably find an old VMDK VMware ubuntu image I was using to play around with thin client firmware from quite a few years back.

  8. @gizmomelb
    Extracting from NAND is useful if you don’t have the firmware, which may happen for some low cost devies in the future with zero support. I got confused when I tried cat /proc/mtd…, but “ls /dev/block/” is what I should have done.

    Now I’m at the stage where I try to boot the rootfs (Linaro ALIP) from NFS. The usual bootargs method does not seem to work at all, so I’m trying switch_root, which sort of work until the “Connection Manager” starts, and seems to mess with the nfsroot…

  9. @m][sko
    Finally even if I can’t mount the NFS share directly with bootargs (I had to edit init with switch_root), it’s still importnt to have a proper bootargs in u-boot. So now I can mount the ALIP rootfs. The display is still black, but at least I get network, and USB mass storage support. I’ll document that tomorrow.

  10. For Rockchip (RK3288) the device tree file is located in resource.img.
    It can be extracted using kernel/resource_img script found in the Android SDK here: http://www.cnx-software.com/2014/09/02/rockchip-rk3288-android-4-4-2-sdk-and-schematics-released-for-firefly-board/ or there: http://www.cnx-software.com/2014/09/16/how-to-build-android-4-4-sdk-for-rk3288-tronsmart-orion-r28-beelink-r89/

    Syntax from kernel folder:

  11. I have a problem: When run the command “split_bootimg.pl boot.img” i’m getting just kernel and ramdisk. I don’t have the second.gz file. What can i do?

  12. I am in the same situation….no boot.img-second.gz file. LG Optimus Fx3Q, Android 4.1.2, kernel version unknown (I am running a custom one, but probably less than 3.7).

    So all it takes is recompiling the kernel against a newer source? Are there any roadblocks/considerations/gremlins associated with such a task?

  13. @Joel
    If there’s no new port, you’d basically need to become a kernel developer,so if you have no experience that would be quite a roadblock, and even if you have that could be months of work.

    If your processor is supported in a more recent version of the kernel, you might just have to write the device tree file by yourself, but that assume all required drivers for your hardware are also part of the kernel. There may also be an issue with binary drivers (GPU. Wi-Fi), and if they don’t match your kernel version, you may be out of luck.

  14. @cnxsoft

    Would merging the kernel source with say, the Optimus F6 (a similar device, which I think has a 3.7+ kernel out there) be any easier? Or is that what you had in mind with my original question?

    I’ve compiled Linux Kernels before (the Debian way), so merging the kernel sources, although daunting, seems less of a chore than writing a device tree by hand (especially when I don’t know where to start). Sadly, although similar, the F6 tree does not work for this device (ROM won’t compile).

    Thanks for your opinion.

  15. @Joel
    If you have a device tree kernel for another device with the same processor, then it will help a lot, but you’ll still need to modify the device tree file yourself, and maybe some other bits too in the source code. It won’t be from scratch so most probably easier. Disclaimer: I’ve never done that myself.

  16. Check and see if the device tree is exposed in /proc. Exposing it is a kernel option.

    Also the dtc compiler is part of the Linux kernel tree. You don’t have to build it separately.

    It is possible to append the DTB onto the end of the kernel image, in that case it is not a separate file. It is still there, just harder to get to.

    Just because these chips are on Linux 3.10 does not force them onto device tree. I have several systems here still using board files on 3.10. I really, really wish these vendors could understand the benefits of contributing code to mainline and then start doing it. A few are starting but many more aren’t.

    Hint to vendors – your pile of out of tree code that has to keep getting ported to each release is only going to growing larger and larger until it collapses and buries you. Once the code is in mainline that part of the pile will stop growing. This is nothing new and has been going on for twenty years. There are hundreds of dead companies out there that collapsed under the ever growing expense of maintaining proprietary Linux versions.

  17. I got the blog has incorrect magic number error.

    dtc -I dtb boot.img-second.gz -O dts -o meson8.dtd
    DTC: dtb->dts on file “boot.img-second.gz”
    FATAL ERROR: Blob has incorrect magic number

  18. cnxsoft :
    I’ve done the same for Tronsmart Vega S89 (Elite). S89 firmware is usually distributed as an IMG file to be used with AML Flash Burning tool, but I haven’t found a way to extract such file yet

    A quick follow up on that, it’s only for Amlogic but i found a way to get your dtb out of an .img update (not ota zip) blob.

    I’m using Amlogic Customization Tool (apparently official tool sourced from some russian forums). It’s purpose is to edit an existing img data file and basically have access to any part of the data, including dtd/dts/dtb.

    Here’s a link to the version of Customization Tool i’ve tested (2.0.7), including a pdf manual which describes how to configure java JDK on your windows (i had to use full paths everywhere as %JAVA_HOME% was not interpreted by my win7).

    https://mega.nz/#F!aIkDUDhJ!UydfamKmRRKhkk6K_W8Apw

    The software is in english, you have to set the first option from the second menu.

    Now when you load the img file you can select various extraction options, “Edit DTD” will give you access to the dtd / dts / dtb files.
    You can directly edit the DTS file, or copy it, you can also generate your “modified” dtb file, it is located in the “tmp” directory of the tool install folder.

    I have taken that “extracted” dtb file and decoded it using dtc tool command as described above, it produced a dts file without errors.

    I also have an OTA firmware (the next version) for the same device (“same” as far as i can tell, as it all comes from forums or blog websites), including the boot.img file.

    So following the process described above i get a second dts file and both are almost identical, only one parameter differs (amlogic,setmask).
    So i would assume the customization tool method is valid. I have not tested it against a kernel on that device, i’m not there yet.

    one thing though, the dts produced directly inside the customization tool, is quite different from the one generated by dtc, but as far as i could tell it mostly concerns hex values being written as decimal and some data arrays being written inline.

    hope it helps.

  19. I try to copy text from the examples, and a bar appears over what I’m trying to copy. I select from below, copy, and it copies a bunch of line numbers. I triple-click to select a line, and the whole thing fades into a different style and discards my selection, so I have to make it again. I try to select another box, the bar appears and moves what I just tried to select down.

    Dude, they’re boxes of text. Stop trying to be more clever than you are with this nonsense. It’s bad enough that you have a wall of about 90 fucking ads without giving people headaches just trying to copy text.

  20. cnxsoft,

    I will like to know, I have three different boxes, two uses the same S802 and one S812 and have Openelec.zip for all and the original .img and .zip. The architecture I reckon are all different ie: Ethernet chip etc.

    I was able to pull the DTD for the M8 from your instructions above from the original K200-ota file fine, and I am sure can do the same for OpenELEC.zip. my question is.

    If I have the LibreELEC.zip how can I put this DTD into it to boot? As I am having bootings issues with LE on the M8.

  21. cnxsoft,

    What I will like you to help me with is how to put this DTD I pulled into the a source code. You showed us how to pull the DTD of any Amlogic FW; now, can you show us how we can put this information into any source code.

  22. @Jeff
    It’s been a while, but if I remember correctly, firmware structure are similar between boards, so I just put it back where I found it in the first firmware.
    If you want to include it in Linux source tree, I think DTS files are located in arch/arm/boot/dts. Something like that.

  23. i have a mtk device and i cant build kernel and our provider doesnt send an ota update how can i do? Can i try fake ota update? Thanks..

  24. I didn’t get this command
    ” Writing boot.img-second.gz … complete. ”

    I got the kernel and ramdisk file. so no dtb file extracted.
    Well. I need these all foe compiling CM13

  25. I didn’t get this command
    ” Writing boot.img-second.gz … complete. ”

    I got the kernel and ramdisk file. so no dtb file extracted.

    Help!

  26. Why my Second size: 0 (0x00000000)?

    Page size: 2048 (0x00000800)
    Kernel size: 4759152 (0x00489e70)
    Ramdisk size: 1690049 (0x0019c9c1)
    Second size: 0 (0x00000000)
    Board name:
    Command line:
    Writing boot.img-kernel … complete.
    Writing boot.img-ramdisk.gz … complete.

    —-
    I used boot.img size 8M from Samsung S3 device

  27. dtb.img file extracted from boot.img this way, not compatible with latest Libreelec, Ubuntu images. Do you know why?

  28. What is the use of this .dts file?
    How to use it to compile kernel without source using a similar device’s kernel source ?

  29. @Tony
    The dts file defined the hardware using text. It needs to be converted to binary (dtb), and you can replace the one for in the firmware for another (similar) device with your own, and it may work.

  30. I’ve been searching high and low for a way to make an 800×480 LCD with HDMI input work with my h96 pro TV stick, and I’ve got an idea that the reason might be that the resolution the LCD needs isn’t defined in the DTS/DTB. Does that sound likely? I suspect the stick has been built just to deal with modern resolutions for TVs, of 720p and above (up to 4K with this one), and it has no information about how to output to a display that reports such a low resolution. I’ve gone through all the other things I can think of, starting with playing around with the wm command, then messing around in build.prop, and I think it has to be something more fundamental… or am I just losing my mind?

    Thanks for any input. Searching for how to overcome this has been extremely frustrating. I keep getting user space solutions that don’t work at all and aren’t relevant to what I’m trying to do. I figure I need to get my hands dirty in the firmware, and thought this might be the place.

  31. cnxsoft :
    @eugene28
    What makes you think the dtb is not compatible?

    Message like
    FATAL ERROR: Blob has incorrect magic number
    makes him think of compatibility…
    There is a problem with a binaries format misunderstanding
    One good man found the reason of magic number inconsistence. (Got from asm code, it looks for 0xD00DFEED and jumps to the error message proc else))
    https://ekasiswanto.wordpress.com/2017/08/16/from-boot-img-to-boot-img-dt-to-dtb-and-dts-in-windows-os/#comment-2171
    There is at least two formats of the DTB binaries. ‘Simply’ DTB and ‘dtb.img’ – a stack of DTB overlays in one binary. Last ones are extracted from the compiled boot images, then should be splitted in to the parts to make DTC process them. If you would like to patch DTB you should assemble the whole structure back again later.
    There are a few fresh DTC versions available in sources in a few repo’s (up to v1.4.6). DTC-tool.exe Windows binary, provided with AmLogic Customization tool is too old (1.2.0), while overlay workarounds started in 2015. I can’t find fresh Win binaries to try, while repo mainainers doesn’t publish them (unfortunately). There is very time and space consuming to collect and install the whole ‘compilation environment’ in Win. It seems to be very excessive to do it just to compile a few console utils… People, who have made all the related fresh binaries – please do publish them all in some reliable place (e,g, XDA attaches, which could be got directly even w/o XDA reg).

  32. Quite confused with the dtb files. Many boot image actually does not have dtb files appended to it, however the burn image contains dtb files. I think maybe the dtb files appended to boot image are probably ignored, instead they are picked up at certain location on the flash. On the amlogic device, there is a /dev/dtb. That’s probably where the dtb image is supposed to be stored.

  33. I was able to split a multi-dtb image into several single dtb images using this tool: https://github.com/Wilhansen/aml-dtbtools

    I’m also very confused with how dtb works with u-boot. There does not seem to be much documentation available. The /dev/dtb does not seem to be a partition but rather provided by the kernel after boot.

    Here are my findings using an amlogic S912 device and playing with u-boot.

    1) Partition information:

    gxm_q201_v1#mmc dev 1
    emmc/sd response timeout, cmd8, status=0x1ff2800
    emmc/sd response timeout, cmd55, status=0x1ff2800
    init_part() 293: PART_TYPE_AML
    [mmc_init] mmc init success
    switch to partitions #0, OK
    mmc1(part 0) is current device
    gxm_q201_v1#mmc part

    Partition Map for MMC device 1 — Partition Type: AML

    Part Start Sect x Size Type name
    00 0 8192 512 U-Boot bootloader
    01 73728 131072 512 U-Boot reserved
    02 221184 1048576 512 U-Boot cache
    03 1286144 16384 512 U-Boot env
    04 1318912 65536 512 U-Boot logo
    05 1400832 65536 512 U-Boot recovery
    06 1482752 16384 512 U-Boot rsv
    07 1515520 16384 512 U-Boot tee
    08 1548288 65536 512 U-Boot crypt
    09 1630208 65536 512 U-Boot misc
    10 1712128 65536 512 U-Boot boot
    11 1794048 6291456 512 U-Boot system
    12 8101888 114040832 512 U-Boot data
    ** Partition 13 not found on device 1 **

    2) Clearing dtb:
    gxm_q201_v1#store erase dtb
    start erase dtb……
    start = 81920,end = 82943
    dev # 1, , several blocks erased OK

    So it seems that dtb information was stored on the mmc at sector 81920 which is the “U-Boot reserved” partition.

    3) boot cmd:
    storeboot=if imgread kernel ${boot_part} ${loadaddr}; then bootm ${loadaddr}; fi;run update;

    So there is no reference to initrd or dtb passed to bootm. However, watching the serial output shows that u-boot loads the dtb into memory and linux kernel is able to pick it up from that same memory address. Not sure exactly how this happens. The bootm documentation says it will pass a bd_info struct to the kernel, I can only assume it retrieves the necessary info from there.

  34. I am doing this on a samsung phone, and wen i run the dtc -I dtb boot.img-second.gz -O dts -o meson8_tm8.dtd command I don’t get a boot.img-second.gz file. My kernel version is 3.18.24. Can you please help me?

Leave a Reply

Your email address will not be published. Required fields are marked *

Boardcon Rockchip and Allwinner SoM and SBC products
Boardcon Rockchip and Allwinner SoM and SBC products