Booting the Nexus 4 from “scratch”

I recently decided I’d like to see if it was possible to boot the Nexus 4 from scratch – the Android build system seems to use a prebuilt bootloader.img.

A few other people have wondered about this, but have since lost interest

If this was a PC I’d be looking at the GRUB source code or similar, but the Nexus 4 was initially a mystery. There’s no proper references to a bootloader in the AOSP source, but there used to be:

A listing of deleted files in GitThe Nexus 4 is significantly newer than the G1, so what the hell is booting Android these days?

For Qualcomm devices and a few others, the answer is the Little Kernel Bootloader, or LK for short. Qualcomm actually publish quite a nice guide for their DragonBoard, which happens to run the same SoC as the Nexus.

After futzing around for a while trying to find source code, I found it in some unknown branch of https://www.codeaurora.org/cgit/quic/la/kernel/lk/ – seriously, I had to run

git log --all -- **/msm8960*

and checkout a commit that looked reasonable:

git branch --contains [hash]

gave me nothing

The next step was to cross compile the lk source for the ARM processor in the Nexus.

git clone --depth=1 https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8

export TOOLCHAIN_PREFIX=arm-eabi-
export PATH=$PATH:/home/ubuntu/arm-eabi-4.8/bin/
cd lk
make msm8960

…which produced quite a bit less than I was expecting

ubuntu@ip-172-30-0-141:~/lk/build-msm8960$ ls -lah
total 8.7M
drwxrwxr-x 9 ubuntu ubuntu 4.0K Jul 31 17:09 .
drwxrwxr-x 15 ubuntu ubuntu 4.0K Jul 31 17:08 ..
drwxrwxr-x 3 ubuntu ubuntu 4.0K Jul 31 17:08 app
-rw-rw-r-- 1 ubuntu ubuntu 282K Jul 31 17:09 appsboot.mbn
-rwxrwxr-x 1 ubuntu ubuntu 282K Jul 31 17:09 appsboot.raw
drwxrwxr-x 3 ubuntu ubuntu 4.0K Jul 31 17:08 arch
-rw-rw-r-- 1 ubuntu ubuntu 1.4K Jul 31 17:08 config.h
drwxrwxr-x 7 ubuntu ubuntu 4.0K Jul 31 17:08 dev
drwxrwxr-x 2 ubuntu ubuntu 4.0K Jul 31 17:08 kernel
drwxrwxr-x 7 ubuntu ubuntu 4.0K Jul 31 17:08 lib
-rwxrwxr-x 1 ubuntu ubuntu 1.7M Jul 31 17:09 lk
-rwxrwxr-x 1 ubuntu ubuntu 282K Jul 31 17:09 lk.bin
-rw-rw-r-- 1 ubuntu ubuntu 4.1M Jul 31 17:09 lk.debug.lst
-rw-rw-r-- 1 ubuntu ubuntu 2.0M Jul 31 17:09 lk.lst
-rw-rw-r-- 1 ubuntu ubuntu 46K Jul 31 17:09 lk.size
-rw-rw-r-- 1 ubuntu ubuntu 87K Jul 31 17:09 lk.sym
-rwxrwxr-x 1 ubuntu ubuntu 18K Jul 31 17:09 mkheader
drwxrwxr-x 4 ubuntu ubuntu 4.0K Jul 31 17:08 platform
-rw-rw-r-- 1 ubuntu ubuntu 2.1K Jul 31 17:09 system-onesegment.ld
drwxrwxr-x 3 ubuntu ubuntu 4.0K Jul 31 17:08 target

Nothing that looks like a bootloader.img there. I know that that particular file should start with “BOOTLDR!”, so let’s check for it.

grep -ir BOOTLDR

Nothing. Shit. Looks like we’ll have to do this the hard way.

If you download the stock images from Google, you’ll notice that they haven’t been changed very often:

897b2c89ab564caf5bb4fe6b90d3526b occam-jdq39/bootloader-mako-makoz10o.img
d56825a3b22b2fe9334b5efa4d117206 occam-jwr66y/bootloader-mako-makoz20i.img
ada04d55afd815674d586a60877587e8 occam-kot49h/bootloader-mako-makoz30d.img
ada04d55afd815674d586a60877587e8 occam-krt16s/bootloader-mako-makoz20i.img
ada04d55afd815674d586a60877587e8 occam-ktu84l/bootloader-mako-makoz30d.img
ada04d55afd815674d586a60877587e8 occam-ktu84p/bootloader-mako-makoz30d.img
a66f24fa47871e6e68c34952a6fb9587 occam-lmy47o/bootloader-mako-makoz30f.img
a66f24fa47871e6e68c34952a6fb9587 occam-lmy47v/bootloader-mako-makoz30f.img
a66f24fa47871e6e68c34952a6fb9587 occam-lrx21t/bootloader-mako-makoz30f.img
a66f24fa47871e6e68c34952a6fb9587 occam-lrx22c/bootloader-mako-makoz30f.img

I decided to concentrate on the makoz30f binary as it’s the same one my Nexus 4 is running. I knew there was an old tool called split_bootimg that could pull Android images apart, so I gave it a shot.

ubuntu@ip-172-30-0-141:~$ perl split_bootimg.pl ~/images/occam-lmy47v/bootloader-mako-makoz30f.img
Android Magic not found in /home/ubuntu/images/occam-lmy47v/bootloader-mako-makoz30f.img. Giving up

Damnit. That tool works on kernel images, which are pretty similar across devices and manufacturers, not bootloaders, which are manufacturer and even device-specific.

I tried staring at split_bootimg’s Perl source and at the bootloader in a hex editor, but nothing immediately stood out, and I’m impatient. To Google!

…which felt a bit like cheating – I Googled “bootloader.img format” and got https://gist.github.com/frederic/cd56923c8af46ae44fd5

ubuntu@ip-172-30-0-141:~$ ./bootunldr ~/images/occam-lmy47v/bootloader-mako-makoz30f.img
#0 sbl1 (94440 bytes)
#1 sbl2 (145448 bytes)
#2 sbl3 (1430160 bytes)
#3 tz (193388 bytes)
#4 rpm (144892 bytes)
#5 aboot (309856 bytes)

048cb7cffe6d1e68d3ed88f2d8fefa31 aboot
aad39fa6f8c661fd21afe01c47f3ddae rpm
1e752230dd972f7d30e5dc508e724842 sbl1
150f5f58c44ebaffb01923beed5260a1 sbl2
890e8e606af016e3ae239af1494821c0 sbl3
1d5c541be1018bb9b6f0183f4c60745c tz

Holy shit, progress!

At this point I noticed a conspicuous lack of source code. LK is the “aboot” part up there, but I didn’t have a way to build any of the others.

Emailing Qualcomm wasn’t a lot of help, because I haven’t won the lottery recently, and I’m allergic to NDAs.

NDA

Having grown tired of large corporations (I also emailed LG), I then rewrote the unpacker in Python, because I don’t trust any C code that I write. Having verified that I got the same binaries as above, I wrote a re-packer that could take the proprietary blobs from Qualcomm and add a Little Kernel/aboot file that I built myself. It can unpack the bootloader.img from the stock image and repack it, and the MD5s even match!

The source is here: https://github.com/voltagex/junkcode/tree/master/Python/nexus4-bootloader-tool

This is where I stopped: I would not recommend anyone create a custom bootloader for the Nexus 4 and attempt to flash it. The reason, and the reason I call this little project a failure is that there’s no easy way to recover the bootloader if it fails to work. The bootloader controls access to adb fastboot, and if that doesn’t work you’re pretty much screwed. See my update for the other problem (oops!)

If you do brick your Nexus 4, your options are:

While I was researching this article I found the following reference boards in varying states of availability:

  • Qualcomm APQ604 reference board
  • Compulab QS600
  • Inforce IF6410

You could probably do a lot of the same testing on these boards and recovery wouldn’t be quite as difficult.
e.g. http://www.compulab.co.il/workspace/mediawiki/index.php5/CM-QS600:Android:_Building_from_source_code#Build_boot_image.28kernel_.2B_initramfs.29

Unfortunately, Inforce requires a purchase before you can download the BSP, CompuLab lists the bootloader source as “coming soon” (and also don’t really want to talk to you unless you are going to buy something)

Addendum: there’s a minor problem with my plans – http://www.reddit.com/r/android/comments/3fcmjz/booting_the_nexus_4_from_scratch/ctnj30v
Basically this means I need another angle of attack. Luckily another Redditor pointed me towards https://github.com/grub4android – there’s no instructions so it looks like I’ll have to write a part 2.

If you want to fund more projects like this, send Bitcoins to 17RugTAi9LdxMUcgVhpWVRRvVsWg11P6V5


If cooking food, rather than ROMs is more your thing, try a recipe from The Cinnamon Scrolls

“Fixing” a buggy Windows driver with a Virtual Machine

Recently, a friend of mine built me a debug cable for the Nexus 4. It turns out that the headphone jack can be turned into a UART cable by applying the right resistance, but that’s a story for another day.

My troubles started as soon as I plugged it in.1-error

So, why is the USB adapter having an existential crisis? The answer seems to be that Prolific really wants you to buy a newer device.

Rebooting into a Linux distro would be an option, but what if you want to work in Windows for whatever reason? There is a “fixed” driver available, but this seems to cause quite a lot of instability for me with BSODs at random intervals.

I knew there was nothing wrong with the device itself, so I turned to virtualization for a solution. VirtualBox has the ability to intercept a USB device on a Windows host and provide it to the guest – but that comes at the cost of the RAM needed to run a whole distro like Debian.

What if I could run just enough to load a USB driver and something like netcat? (which can be used to get data to and from a serial port)

Enter Buildroot. It’s a set of scripts which can build all kinds of embedded system images – it’s most commonly used to build modem filesystems for MIPS and ARM based devices but it can also be used to create images that will work on x86.

I was able to hack a quick version of this together but after talking with people in #buildroot on Freenode, I spent some time learning how to use the build system properly. There’s a way to store all of your configuration changes separately from the buildroot code itself, described in Building Out of Tree in the manual. With a lot of tinkering, I now have a git repo that can be checked out and rebuilt very easily.

I worked on this for about a week and a half and now have an image that can be booted in VirtualBox with the PL2302 drivers loaded automagically. This means I can let VirtualBox take over the serial device and it won’t crash Windows.

I also wrote a script to take an empty image created with dd, partition and format it, then make it bootable with extlinux and add my working kernel/initramfs to it.

When I tested my first versions of this it would boot with about 28mb of RAM allocated to the VM, but let’s say the minimum is 32mb to be safe, which is small enough to be run on my laptop without impacting performance.

Building this system went from a couple of hours work to a week of troubleshooting and asking questions in IRC. It’s probably bigger than my goal now and in retrospect maybe I should have just dual booted to work with the serial converter instead. Although, if I’d done that I wouldn’t know buildroot nearly as well as I know it now.

Linux booting and detecting a USB to serial converter

Picocom and microcom are installed in this tiny system, so before networking it up I tested it out with

microcom -s 115200 /dev/ttyUSB0

and… nothing. Panic – had the soldering on the cable held up to being packed in a suitcase? What had I done wrong? I installed the Windows driver mentioned above and connected via PuTTY – also nothing. Not even line noise at 9600 baud. Just before I was going to give up for the day, I noticed that the Nexus 4 bumper case was stopping the cable going all the way in. With the case gone, the cable and serial adapter worked fine and my job was done.

I haven’t tested multidirectional communication yet, I don’t think the Nexus 4 is accepting input on the UART but the following commands let me connect to 192.168.56.101 on port 1234 and read the console output

stty -F /dev/ttyUSB0 raw
stty -F /dev/ttyUSB0 115200
udhcpc
nc -l -p 1234 < /dev/ttyUSB0

putty

Mounting a Windows 8 (or 7 or 10) ISO on Linux

Recently I had to rescue a Windows system and required a new bootable USB stick.

Mounting the Windows 8.1 installation ISO via the GUI brought up an empty folder or a blank README.txt.

I found the solution via superuser.com – the mount helper isn’t detecting the UDF partition on the disk, so you’ll have to do something via the command line like:

mount -o loop -t udf /media/files/en_windows_8.1_with_update_x64_dvd_4065090.iso /tmp/iso

Say your USB stick is on /media/usb and already formatted with FAT32 (vfat), then you can do

rsync -av ---progress /tmp/iso/. /media/usb/.

and you’ll get a UEFI-bootable Windows 8 installer (that can also be used to get into a recovery environment).