How to Build/Run Android Cuttlefish Emulator on PC/ARM64

Cuttlefish is new virtual-machine based Android emulator. It uses virtio devices instead of emulated devices as in original Android emulator.   As such, it needs lighter VM support (to the extent it can run on ARM64 host), unlike Android Emulator which requires heavily modified QEMU to emulate various devices.  The virtio architecture can potentially offer better performance as well.

Refer to  a slide deck on cuttlefish.  Or find a local copy of it at here.

Many thanks to Alistair Delva from Google, who provided many technical guidance in going through this exercise.

How to Build/Run Cuttlefish on PC (X86_64)

My host Ubuntu 18.04.  Refer to https://source.android.com/setup/build/building

Build and install cuttlefish-common package

• checkout code from https://github.com/google/android-cuttlefish

git clone https://github.com/google/android-cuttlefish.git 
cd android-cuttlefish
dpkg-buildpackage --no-sign

• Install :

dpkg -i  ../cuttlefish-common_0.9.9_amd64.deb

• it requires dnsmasq-base and a few other packages; install them as requested
• Check status  with  /etc/init.d/cuttlefish-common status

Build cuttlefish

• Checkout AOSP pie-gsi branch.

repo init -u https://android.googlesource.com/platform/manifest -b pie-gsi repo sync -j 8
source build/envsetup.sh
lunchaosp_cf_x86_64_phone-userdebug
make

Run cuttlefish

• Add user to proper groups before running; power off machine; then power on again.  (Strange, rebooting did not seem to work somehow).  You may need to run “sudo apt install qemu-kvm” if you get error, “group kvm doesn’t exist”.

sudo usermod -aG kvm $USER
sudo usermod -aG cvdnetwork $USER

• Run: launch_cvd
• Install tightvnc to view phone
  • download tightvnc viewer (jar file): https://www.tightvnc.com/download.php
  • install java if not done yet:  sudo apt install openjdk-11-jre
  • java -jar tightvnc-jviewer.jar  #use 127.0.0.1:6444
• Run stop_cvd to kill the cvd

How to Build/Run Cuttlefish on ARM64

My ARM64 board is rockpro64, running ubuntu 18.04.

[on X86_64] Cross-build cuttlefish for ARM64

• Similar as above, except build with a different target and with distribution packages

lunch aosp_cf_arm64_phone-userdebug
make dist

Note the following output files, which need to be copied to ARM64 host

out/dist/aosp_cf_arm64_phone-img-eng.jsun.zip
out/dist/cvd-host_package.tar.gz

[on X86_64] Configure and build arm64 kernel

You will need CONFIG_BINFMT_MISC.  Otherwise below step will fail.  Check /proc/sys/fs/binfmt_misc to be sure.

In addition, you will need a few other kernel configs, which according to Alistair are only supported in kernel after 4.9.  Here is the set of configs I added to rockpro64 default v5.2 kernel.

CONFIG_BINFMT_MISC=y
CONFIG_EVENTFD=y
CONFIG_VSOCKETS=y
CONFIG_VHOST_NET=m
CONFIG_VHOST_SCSI=m
CONFIG_VHOST_VSOCK=m
CONFIG_VHOST=m
CONFIG_VIRTIO_BLK_SCSI=m
CONFIG_VIRTIO_INPUT=m
CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES=y
CONFIG_VIRTIO_VSOCKETS_COMMON=m
CONFIG_VIRTIO_VSOCKETS=m

In specific, here are the exact commands I used to build my rockpro64 kernel on my PC ubuntu:

git clone https://github.com/ayufan-rock64/linux-mainline-kernel.git
cd linux-mainline-kernel/
git checkout -b 5.2.0-1116-ayufan-js 5.2.0-1116-ayufan
vi arch/arm64/configs/rockchip_linux_defconfig    # add the above configs to the end
vi dev.mk   # BUG? change HOSTCC=aarch64-linux-gnu-gcc to HOSTCC=gcc
./dev-make kernel-image-and-modules
./dev-make kernel-package

Copy over the .deb package file to arm64 host and install with “dpkg -i <pkg file>” command.  Reboot afterwards.

[on ARM64] Setup to run x86_64 binaries

Do following as root user:

apt install qemu-user-static
dpkg --add-architecture amd64
[You may need to correct /etc/apt/source.list file here.  See an example at this link]
apt install libc6:amd64

After this you should be able to run simple x86_64 binaries such as “cat”.  Give it a try.

[on ARM64] Build and install cuttlefish-common package

This is similar to x86_64 case, except that you do this step on ARM64 host.

The following packages are needed before you can install cuttlefish-common:

apt install bridge-utils libarchive-tools libfdt1 python iptables

[on ARM64] Setup and run cuttlefish

• copy the following  the files from x86_64 PC host:

out/dist/aosp_cf_arm64_phone-img-eng.jsun.zip
out/dist/cvd-host_package.tar.gz

• untar and unzip them into 2 directories, say /home/jsun/work/cuttlefish/host and /home/jsun/work/cuttlefish/image
• Add user to proper groups before running; power off machine; then power on again.  You may need to create kvm group first and make sure  /dev/kvm is read/writable by kvm group

sudo usermod -aG kvm $USER
sudo usermod -aG cvdnetwork $USER

• Run the following commands to start cuttlefish

export ANDROID_PRODUCT_OUT=/home/jsun/work/cuttlefish/image/
export ANDROID_HOST_OUT=/home/jsun/work/cuttlefish/host/
export PATH=$PATH:$ANDROID_HOST_OUT/bin
launch_cvd -decompress_kernel=true

• The rest are similar to x86_64 case

Appendix – Install Ubuntu Desktop on ARM64

Many ARM64 ubuntu distros are minimal or server, which means no desktop included.  It is easy to install one.  However, without a few key steps (see first a few commands below),  you can easily get some headaches.

Below are the commands I used to install Xubuntu on rockpro64, starting from their minimal Ubuntu 18.04 distro.

1. Download Ubuntu 18.04 minimal img for SD card (refer to this page),
2. Copy to SD card

dd if=./bionic-minimal-rockpro64-0.8.3-1141-arm64.img of=/dev/sdb bs=4M

1. Use parted or gparted to expand /dev/sdb7 to take over whole SD space
2. Boot up rockpro64:

sudo su
locale-gen
localectl set-locale LANG="en_US.UTF-8"
apt update
apt install -y xubuntu-desktop