Wednesday, August 13, 2014

Booting Linaro ARMv8 OE images with Qemu

A quick update - Linaro ARMv8 OpenEmbbeded images work just fine with qemu 2.1 as well:
$ http://releases.linaro.org/14.07/openembedded/aarch64/Image
$ http://releases.linaro.org/14.07/openembedded/aarch64/vexpress64-openembedded_lamp-armv8-gcc-4.9_20140727-682.img.gz
$ qemu-system-aarch64 -m 1024 -cpu cortex-a57 -nographic -machine virt \
 -kernel Image -append 'root=/dev/vda2 rw rootwait mem=1024M console=ttyAMA0,38400n8' \
 -drive if=none,id=image,file=vexpress64-openembedded_lamp-armv8-gcc-4.9_20140727-682.img \
 -netdev user,id=user0 -device virtio-net-device,netdev=user0  -device virtio-blk-device,drive=image 
[    0.000000] Linux version 3.16.0-1-linaro-vexpress64 (buildslave@x86-64-07) (gcc version 4.8.3 20140401 (prerelease) (crosstool-NG linaro-1.13.1-4.8-2014.04 - Linaro GCC 4.8-2014.04) ) #1ubuntu1~ci+140726114341 SMP PREEMPT Sat Jul 26 11:44:27 UTC 20
[    0.000000] CPU: AArch64 Processor [411fd070] revision 0
...
root@genericarmv8:~# 
Quick benchmarking with age-old ByteMark nbench:
Index Qemu Foundation Host
Memory 4.294 0.712 44.534
Integer 6.270 0.686 41.983
Float 1.463 1.065 59.528
Baseline (LINUX) : AMD K6/233*
Qemu is upto 8x faster than Foundation model on Integers, but only 50% faster on Math. Meanwhile, the Host pc spends 7-40x slower emulating ARMv8 than executing native instructions.

Tuesday, August 5, 2014

Testing qemu 2.1 arm64 support

Qemu 2.1 was just released a few days ago, and is now a available on Debian/unstable. Trying out an (virtual) arm64 machine is now just a few steps away for unstable users:
$ sudo apt-get install qemu-system-arm 
$ wget https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-arm64-disk1.img
$ wget https://cloud-images.ubuntu.com/trusty/current/unpacked/trusty-server-cloudimg-arm64-vmlinuz-generic
$ qemu-system-aarch64 -m 1024 -cpu cortex-a57 -nographic -machine virt -kernel trusty-server-cloudimg-arm64-vmlinuz-generic \ 
-append 'root=/dev/vda1 rw rootwait mem=1024M console=ttyAMA0,38400n8 init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=randomstring' \
-drive if=none,id=image,file=trusty-server-cloudimg-arm64-disk1.img \
-netdev user,id=user0 -device virtio-net-device,netdev=user0 -device virtio-blk-device,drive=image 
[    0.000000] Linux version 3.13.0-32-generic (buildd@beebe) (gcc version 4.8.2 (Ubuntu/Linaro 4.8.2-19ubuntu1) ) #57-Ubuntu SMP Tue Jul 15 03:52:14 UTC 2014 (Ubuntu 3.13.0-32.57-generic 3.13.11.4)
[    0.000000] CPU: AArch64 Processor [411fd070] revision 0
...
-snip-
...
ubuntu@ubuntu:~$ cat /proc/cpuinfo 
Processor       : AArch64 Processor rev 0 (aarch64)
processor       : 0
Features        : fp asimd evtstrm 
CPU implementer : 0x41
CPU architecture: AArch64
CPU variant     : 0x1
CPU part        : 0xd07
CPU revision    : 0

Hardware        : linux,dummy-virt
ubuntu@ubuntu:~$
The "init=/usr/lib/cloud-init/uncloud-init ds=nocloud ubuntu-pass=randomstring" is ubuntu cloud stuff that will set the ubuntu user password to "randomstring" - don't use "randomstring" literally there, if you are connected to internets...

For more detailed writeup of using qemu-system-aarch64, check the excellent writeup from Alex Bennee.

Thursday, May 8, 2014

Arm builder updates

Debian has recently received a donation of 8 build machines from Marvell. The new machines come with Quad core MV78460 Armada XP CPU's, DDR3 DIMM slot so we can plug in more memory, and speedy sata ports. They replace the well served Marvell MV78200 based builders - ones that have been building debian armel since 2009. We are planning a more detailed announcement, but I'll provide a quick summary:

The speed increase provided by MV78460 can viewed by comparing build times on selected builds since early april:

Qemu build times.

We can now build Qemu in 2h instead of 16h -8x faster than before! Certainly a substantial improvement, so impressive kit from Marvell! But not all packages gain this amount of speedup:

webkitgtk build times.

This example, webkitgtk, builds barely 3x faster. The explanation is found from debian/rules of webkitgkt:

# Parallel builds are unstable, see #714072 and #722520
# ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
#   NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
#   MAKEARGUMENTS += -j$(NUMJOBS)
# endif
The old builders are single-core[1], so the regardless of parallel building, you can easily max out the cpu. New builders will use only 1 of 4 cores without parallel build support in debian/rules.

During this buildd cpu usage graph, we see most time only one CPU is consumed. So for fast package build times.. make sure your packages supports parallel building.

For developers, abel.debian.org is porter machine with Armada XP. It has schroot's for both armel and armhf. set "DEB_BUILD_OPTIONS=parallel=4" and off you go.

Finally I'd like to thank Thomas Petazzoni, Maen Suleiman, Hector Oron, Steve McIntyre, Adam Conrad and Jon Ward for making the upgrade happen.

Meanwhile, we have unrelated trouble - a bunch of disks have broken within a few days apart. I take the warranty just run out...

[1] only from Linux's point of view. - mv78200 has actually 2 cores, just not SMP or coherent. You could run an RTOS on the other core while you run Linux on the other.

Friday, February 21, 2014

Where the armel buildd time went

Wanna-build, wanna-build, which packages spent most time on armel buildd's since beginning of 2013?
         package         | sum(build_time)
-------------------------+--------------
 libreoffice             | 114 09:16:34
 linux                   | 113 02:58:50
 gcc-4.8                 | 064 01:21:09
 webkitgtk               | 059 19:09:27
 acl2                    | 043 16:40:50
 gcc-4.7                 | 028 14:03:53
 iceweasel               | 026 19:02:13
 gcc-snapshot            | 026 01:31:21
 openjdk-7               | 020 02:41:53
 php5                    | 019 16:13:22
 llvm-toolchain-3.3      | 017 19:05:38
 qt4-x11                 | 017 02:57:09
 espresso                | 016 03:50:37
 pypy                    | 015 07:07:25
 icedove                 | 014 18:57:08
 insighttoolkit4         | 014 17:16:43
 qtbase-opensource-src   | 014 12:39:09
 llvm-toolchain-3.4      | 012 03:06:15
 mono                    | 011 22:30:13
 atlas                   | 011 20:40:54
 qemu                    | 011 17:11:09
 calligra                | 011 16:05:55
 gnuradio                | 011 15:19:35
 resiprocate             | 011 10:14:56
 llvm-toolchain-snapshot | 011 02:04:44
 libav                   | 010 13:52:03
 python2.7               | 009 18:58:33
 ghc                     | 009 18:28:48
 gnat-4.8                | 009 13:59:57
 axiom                   | 009 12:40:24
 cython                  | 009 00:47:04
 openjdk-6               | 008 16:38:14
 oce                     | 008 10:29:20
 eglibc                  | 008 06:04:26
 ppl                     | 007 20:48:45
 root-system             | 007 17:32:16
 openturns               | 007 10:12:53
 gcl                     | 007 08:02:42
 gcc-4.6                 | 007 02:50:48
 k3d                     | 007 00:36:11
 python3.3               | 007 00:25:42
 llvm-toolchain-3.2      | 007 00:17:59
 vtk                     | 006 17:53:28
 samba                   | 006 17:17:27
 mysql-workbench         | 006 14:36:46
 kde-workspace           | 006 07:31:12
 gmsh                    | 006 04:32:42
 psi-plus                | 006 04:30:08
 octave                  | 006 04:17:22
 paraview                | 006 04:13:25
Timeformat is "days HH:MM:SS". Our ridiculously stable mv78x00 buildd's have served well, but has come to become let them rest. Now, to find out how many of these top time consuming packages can build with parallel make and are not doing so already.

Friday, December 20, 2013

Replicant on Galaxy S3

I recently got my self and Galaxy S3 for testing out Replicant, an android image made out of only open source components.

Why Galaxy S3?

It is well supported in Replicant, almost every driver is already open source. The hardware specs are acceptable, 1.4Ghz quad core, 1GB ram, microsd, and all the peripheral chips one expects for a phone. Galaxy S3 has sold insanely (50 million units supposedly), meaning I won't run out of accessories and aftermarket spare parts any time soon. The massive installed base also means a huge potential user community. S3 is still available as new, with two years of warranty.

Why not

While the S3 is still available new, it is safe to assume production is ending already - 1.5 year old product is ancient history in mobile world! It remains to be seen how much the massive user base will defend against the obsolescence. Upstream kernel support for "old" cpu is open question, replicant is still basing kernel on vendor kernel. Bootloader is unlocked, but it can't be changed due to trusted^Wtreacherous computing, preventing things like boot from sd card. Finally, not everything is open source, the GPU (mali) driver while being reverse engineered, is taking it's time - and the GPS hasn't been reversed yet.

Installing replicant

Before install, from the original installation, you might want to take a copy of firmware files (since replicant won't provide them). enable developer mode on the S3 and:
sudo apt-get install android-tools
mkdir firmware
adb pull /system/vendor/firmware/  
adb pull /system/etc/wifi
After then, just follow official replicant install guide for S3. If you don't mind closed source firmwares, post-install you need to push the firmware files back:
adb shell
mount -o remount,rw /system
adb push . /system/vendor/firmware
Here was my first catch, the wifi firmwares from jelly bean based image were not compatible with older ICS based replicant.

Using replicant

Booting to replicant is fast, few seconds to the pin screen. You are treated with the standard android lockscreen, usual slide/pin/pattern options are available. Basic functions like phone, sms and web browsing have icons from the homescreen and work without a hitch. Likewise camera seems to work, really the only smartphone feature missing is GPS.

Sidenote - this image looks a LOT better on the S3 than on my thinkpad. No wonder people are flocking to phones and tablets when laptop makers use such crappy components.

The grid menu has the standard android AOSP opensource applications in the ICS style menu with the extra of f-droid icon - which is the installer for open source applications. F-droid is it's own project that complements replicant project by maintaining a catalog of Free Software.
F-droid brings hundreds of open source applications not only for replicant, but for any other android users, including platforms with android compatibility, such as Jolla's Sailfish OS. Of course f-droid client is open source, like the f-droid server (in Debian too). F-droid server is not just repository management, it can take care of building and deploying android apps.
The WebKit based android browser renders web sites without issues, and if you are not happy with, you can download Firefox from f-droid. Many websites will notice you are mobile, and provide mobile web sites, which is sometimes good and sometimes annoying. Worse, some pages detect you are android and only offer you to load their closed android app for viewing the page. OTOH I am already viewing their closed source website, so using closed source app to view it isn't much worse.

This keyboard is again the android standard one, but for most unixy people the hacker's keyboard with arrow buttons and ctrl/alt will probably be the one you want.

Closing thoughts

While using replicant has been very smooth, the lack of GPS is becoming a deal-breaker. I could just copy the gpsd from cyanogen, like some have done, but it kind of beats the purpose of having replicant on the phone. So it might be that I move back to cyanogen, unless I find time to help reverse engineering the BCM4751 GPS.

Monday, July 22, 2013

ACPI on ARM storm in teacup

A recent google+ post by Jon Masters caused some stormy and some less stormy responses.

A lot of BIOS/UEFI/ACPI hate comes X86, where ACPI is used from everything from suspending devices to reading buttons and setting leds. So when X86 kernel suspends, it does magic calls to ACPI and prays that the firmware vendor did not screw it up. Now vendors do screw up, hence lots of cursing and ugly workarounds in the kernel follows. My Lenovo has a firwmare bug where the FN-buttons and Fan stops working if the laptop is attached on AC adapter for too long. The fan is probably a simple i2c device the kernel could control directly without jumping through ACPI hiding layer hoops. But the X86 people hold the view it is better to trust the firmware engineer to control devices instead of having the kernel folk to write device drivers to ... control devices!

Now on ARM(64) the idea of using ACPI is to have none of that.

Instead the idea is to use ACPI only to provide tables for enumerating what devices are available in the platform. Just like what device tree does. Now if this is the same as device tree, why bother?

The main reason is to allow the distribution installer behave same on X86 / ARM / ARM64. This is crucial for distributions like fedora and RHEL where a cabal holds the point of view that X86 distribution development must not be constrained by ARM support. But it also important for everyone that the method of installing your favorite distribution to an ARM64 server is standard and works the same for any server from any vendor. Now while UEFI and ACPI are definitely not my preferred solutions, I can accept them as necessary evil for having a more standard platform.

Monday, February 4, 2013

On behalf of aarch64 porters

Public service announcement

When porting GNU/Linux applications to a new architecture, such as 64-Bit ARM, one gets familiar with the following error message:

checking build system type... x86_64-pc-linux-gnu
checking host system type... Invalid configuration `aarch64-oe-linux': machine `aarch64-oe' not recognized
configure: error: /bin/sh config.sub aarch64-oe-linux failed

This in itself is trivial to fix - run autoreconf or just copy in new versions of config.sub and config.guess. However, when bootstrapping a distribution of 12000+ packages, this becomes quickly tiresome. Thus we have a small request:

If you are an upstream of a software that uses autoconf - Please run autoreconf against autotools-dev 20120210.1 or later, and make a release of your software.

Aarch64 porters will be grateful as updated software trickles down to distributions.

This was the most discussed point during my FOSDEM talk "Porting applications to 64-Bit ARM".