Showing posts with label qemu. Show all posts
Showing posts with label qemu. Show all posts

Monday, May 9, 2016

Booting ubuntu 16.04 cloud images on Arm64

For testing kvm/qemu, prebaked images cloud images are nice. However, there is a few steps to get started. First we need a recent Qemu (2.5 is good enough). An efi firmware is needed, and cloud-utils, for customizing our VM.
sudo apt install -y qemu qemu-utils cloud-utils
wget https://releases.linaro.org/components/kernel/uefi-linaro/15.12/release/qemu64/QEMU_EFI.fd
wget https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-arm64-uefi1.img
Cloud images are plain - there is no user setup, no default user/pw combo, so to log in to the image, we need to customize the image on first boot. The defacto tool for this is cloud-init. The simplest method for using cloud-init is passing a block media with a settings file - of course for real cloud deployment, you would use one of fancy network based initialization protocols cloud-init supports. Enter the following to a file, say cloud.txt:
#cloud-config

users:
  - name: you
    ssh-authorized-keys:
      - ssh-rsa AAAAB3Nz....
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    groups: sudo
    shell: /bin/bash
This minimal config will just set you a user with ssh key. A more complex setup can install packages, write files and run arbitrary commands on first boot. In professional setups, you would most likely end up using cloud-init only to start Ansible or another configuration management tool.
cloud-localds cloud.img cloud.txt
qemu-system-aarch64 -smp 2 -m 1024 -M virt -bios QEMU_EFI.fd -nographic \
       -device virtio-blk-device,drive=image \
       -drive if=none,id=image,file=xenial-server-cloudimg-arm64-uefi1.img \
       -device virtio-blk-device,drive=cloud \
       -drive if=none,id=cloud,file=cloud.img \
       -netdev user,id=user0 -device virtio-net-device,netdev=user0 -redir tcp:2222::22 \
       -enable-kvm -cpu host 
If you are on an X86 host and want to use qemu to run an aarch64 image, replace the last line with "-cpu cortex-a57". Now, since the example uses user networking with tcp port redirect, you can ssh into the VM:
ssh -p 2222 you@localhost
Welcome to Ubuntu 16.04 LTS (GNU/Linux 4.4.0-22-generic aarch64)
....

Saturday, November 1, 2014

Using networkd for kvm tap networking

Setting up basic systemd-network was recently described by Joachim, and the post inspired me to try it as well. The twist is that in my case I need a bridge for my KVM with Lava server and arm/aarch64 qemu system emulators...

For background, qemu/kvm support a few ways to provide network to guests. The default is user networking, which requires no privileges, but is slow and based on ancient SLIRP code. The other common option is tap networking, which is fast, but complicated to set up. Turns out, with networkd and qemu bridge helper, tap is easy to set up.

$ for file in /etc/systemd/network/*; do echo $file; cat $file; done
/etc/systemd/network/eth.network
[Match]
Name=eth1
[Network]
Bridge=br0

/etc/systemd/network/kvm.netdev
[NetDev]
Name=br0
Kind=bridge

/etc/systemd/network/kvm.network
[Match]
Name=br0
[Network]
DHCP=yes

Diverging from Joachims simple example, we replaced "DHCP=yes" with "Bridge=br0". Then we proceed to define the bridge (in the kvm.netdev) and give it an ip via dhcp in kvm.network. From the kvm side, if you haven't used the bridge helper before, you need to give the helper permissions (setuid root or cap_net_admin) to create a tap device to attach on the bridge. The helper needs an configuration file to tell what bridge it may meddle with.
# cat > /etc/qemu/bridge.conf <<__END__
allow br0
__END__
# setcap cap_net_admin=ep /usr/lib/qemu/qemu-bridge-helper
Now we can start kvm with bridge networking as easily as with user networking:
$ kvm -m 2048 -drive file=jessie.img,if=virtio -net bridge -net nic,model=virtio -serial stdio
The manpages systemd.network(5) and systemd.netdev(5) do a great job explaining the files. Qemu/kvm networking docs are unfortunately not as detailed.

Wednesday, August 13, 2014

Booting Linaro ARMv8 OE images with Qemu

A quick update - Linaro ARMv8 OpenEmbbeded images work just fine with qemu 2.1 as well:
$ http://releases.linaro.org/14.07/openembedded/aarch64/Image
$ http://releases.linaro.org/14.07/openembedded/aarch64/vexpress64-openembedded_lamp-armv8-gcc-4.9_20140727-682.img.gz
$ qemu-system-aarch64 -m 1024 -cpu cortex-a57 -nographic -machine virt \
 -kernel Image -append 'root=/dev/vda2 rw rootwait mem=1024M console=ttyAMA0,38400n8' \
 -drive if=none,id=image,file=vexpress64-openembedded_lamp-armv8-gcc-4.9_20140727-682.img \
 -netdev user,id=user0 -device virtio-net-device,netdev=user0  -device virtio-blk-device,drive=image 
[    0.000000] Linux version 3.16.0-1-linaro-vexpress64 (buildslave@x86-64-07) (gcc version 4.8.3 20140401 (prerelease) (crosstool-NG linaro-1.13.1-4.8-2014.04 - Linaro GCC 4.8-2014.04) ) #1ubuntu1~ci+140726114341 SMP PREEMPT Sat Jul 26 11:44:27 UTC 20
[    0.000000] CPU: AArch64 Processor [411fd070] revision 0
...
root@genericarmv8:~# 
Quick benchmarking with age-old ByteMark nbench:
Index Qemu Foundation Host
Memory 4.294 0.712 44.534
Integer 6.270 0.686 41.983
Float 1.463 1.065 59.528
Baseline (LINUX) : AMD K6/233*
Qemu is upto 8x faster than Foundation model on Integers, but only 50% faster on Math. Meanwhile, the Host pc spends 7-40x slower emulating ARMv8 than executing native instructions.

Tuesday, August 5, 2014

Testing qemu 2.1 arm64 support

Qemu 2.1 was just released a few days ago, and is now a available on Debian/unstable. Trying out an (virtual) arm64 machine is now just a few steps away for unstable users:
$ sudo apt-get install qemu-system-arm 
$ wget https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-arm64-disk1.img
$ wget https://cloud-images.ubuntu.com/trusty/current/unpacked/trusty-server-cloudimg-arm64-vmlinuz-generic
$ qemu-system-aarch64 -m 1024 -cpu cortex-a57 -nographic -machine virt -kernel trusty-server-cloudimg-arm64-vmlinuz-generic \ 
-append 'root=/dev/vda1 rw rootwait mem=1024M console=ttyAMA0,38400n8 init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=randomstring' \
-drive if=none,id=image,file=trusty-server-cloudimg-arm64-disk1.img \
-netdev user,id=user0 -device virtio-net-device,netdev=user0 -device virtio-blk-device,drive=image 
[    0.000000] Linux version 3.13.0-32-generic (buildd@beebe) (gcc version 4.8.2 (Ubuntu/Linaro 4.8.2-19ubuntu1) ) #57-Ubuntu SMP Tue Jul 15 03:52:14 UTC 2014 (Ubuntu 3.13.0-32.57-generic 3.13.11.4)
[    0.000000] CPU: AArch64 Processor [411fd070] revision 0
...
-snip-
...
ubuntu@ubuntu:~$ cat /proc/cpuinfo 
Processor       : AArch64 Processor rev 0 (aarch64)
processor       : 0
Features        : fp asimd evtstrm 
CPU implementer : 0x41
CPU architecture: AArch64
CPU variant     : 0x1
CPU part        : 0xd07
CPU revision    : 0

Hardware        : linux,dummy-virt
ubuntu@ubuntu:~$
The "init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=randomstring" is ubuntu cloud stuff that will set the ubuntu user password to "randomstring" - don't use "randomstring" literally there, if you are connected to internets...

For more detailed writeup of using qemu-system-aarch64, check the excellent writeup from Alex Bennee.

Friday, April 27, 2012

Cross Compiling with MultiArch

Congrats to the Ubuntu Folk for the new LTS release. Incidentally this is also the first release where our work on MultiArch bears fruit. We can now cross-compile relatively easily without resorting to hacks like scratchbox, qemu chroot or dpkg-cross/xdeb.

Lets show short practical guide on cross-building Qemu for armhf. The instructions refer to precise, but the goodiness is on the way to Debian as well. Biggest missing piece being the cross-compiler packages, which we have an Summer of Code project. The example is operated in a chroot to avoid potentially messing up your working machine. Qemu-linaro is not a shining example, as cross-building it doesn't work out of box. But that is educational, it allows me to show what kind of issues you can bump into, and how to fix them.

 $ sudo debootstrap --variant=buildd precise /srv/precise
Edit the /srv/precise/etc/apt.sources.list to the following (replace amd64 with i386 if you made an i386 chroot)
deb [arch=amd64] http://archive.ubuntu.com/ubuntu precise main universe
deb [arch=armhf] http://ports.ubuntu.com/ubuntu-ports precise main universe
deb-src http://archive.ubuntu.com/ubuntu precise main universe
Edit the /srv/precise/etc/dpkg/dpkg.cfg.d/multiarch by adding the following line:
foreign-architecture armhf
Finally disable install of recommends by editing /srv/precise/etc/apt/apt.conf.d/10local:
APT::Install-Recommends "0";
APT::Install-Suggests "0";
Install the armhf crosscompiler the chroot
 $ sudo chroot /srv/precise/
 # unset LANG LANGUAGE
 # mount -t proc proc /proc
 # apt-get update
 # apt-get install g++-arm-linux-gnueabihf pkg-config-arm-linux-gnueabihf
Get the sources and try to install the cross-buildeps:
 # cd /tmp
 # apt-get source qemu-linaro
 # cd qemu-linaro-*
 # apt-get build-dep -aarmhf qemu-linaro
As we see, the build-dep bombs out ugly, with a them of "perl" being unable to be installed. This is because apt-get can't figure out if we should install and armhf or amd64 version of perl. We don't use the required syntax yet in Build-Dep line, "perl:any", as dpkg and apt in previous released don't support it. Thus back porting would no longer be possible. One way to fix it, would be to drop perl build-dep, as perl is already pulled by other build-deps. But lets instead show howto install it the build-deps manually. First we build system build-deps, then target architecture ones:
 # apt-get install debhelper texinfo
 # apt-get install zlib1g-dev:armhf libasound2-dev:armhf libsdl1.2-dev:armhf libx11-dev:armhf libpulse-dev:armhf libncurses5-dev:armhf libbrlapi-dev:armhf libcurl4-gnutls-dev:armhf libgnutls-dev:armhf libsasl2-dev:armhf uuid-dev:armhf libvdeplug2-dev:armhf libbluetooth-dev:armhf
And try the build[1]:
 # dpkg-buildpackage -aarmhf -B
Which sadly errors out. Turns out the cross-build support in debian/rules is broken. Instead of --cc we need to feed an --cross-prefix to the ./configure of qemu. Edit the debian/rules with replacing
-  conf_arch += --cc=$(DEB_HOST_GNU_TYPE)-gcc --cpu=$(QEMU_CPU)
+  conf_arch += --cross-prefix=$(DEB_HOST_GNU_TYPE)-
Optional: Since we are cross-compiling from a multicore machine, lets also add parallel building support, by changing the override_dh_auto_build: rule to have --parallel flags in debian/rules as well:
override_dh_auto_build:
        # system build
        dh_auto_build -B system-build --parallel
ifeq ($(DEB_HOST_ARCH_OS),linux)
        # user build
        dh_auto_build -B user-build --parallel

        # static user build
        dh_auto_build -B user-static-build --parallel

Try the build again:
 # export DEB_BUILD_OPTIONS=parallel=4 # I have dual-core hyperthreading machine, build with all four threads
 # time dpkg-buildpackage -aarmhf -B
 ...
dpkg-buildpackage: binary only upload (no source included)

real    18m53.425s
user    65m42.570s
sys     2m50.383s
The Native build of qemu-linaro took 4h 11min.

[1] You should not build as root - but to keep instructions short, I'm not explaining howto add and use a unprivileged user in a chroot. Do as I *say*, not as I *do*!