For the past couple of days I’ve been attending the first ever sprint held by the Linaro Tools and Infrastructure team. The sprint itself lasts all week but I arranged to be there for just Tuesday and Wednesday to find out what they had planned and to join up with a few guests.

This event is slightly different from what we typically do at Ubuntu sprints. The team hasn’t been together for that long so there was equal focus on both information sharing and collaboration. To that end, mornings were for presentations and discussion, afternoons for hands-on hacking.

A link to the whole schedule is here; for my account of what happened whilst I was there, read on.

Tuesday started with a discussion on image building by Michael Hudson. Often lively and full of good information, the key steps to producing the Linaro images was the focus of conversation. Two launchpad specifications cover this effort, the first, arm-m-image-building-tool concerns improving live-helper to accommodate Linaro’s needs, the second, arm-m-image-building-console aims to produce a web based front-end for the image building tools.

Zygmunt Krynicki’s project entitled Validation Dashboard was next (slides can be found here). An ambitious project to produce a front end visual representation for various testing tools; this effort is of great importance as validation and correctness is paramount to the software releases Linaro makes.

Another requirement for Linaro is the ability to branch and rebuild parts of the Ubuntu archive. Linaro benefits from the great effort that is put into the Ubuntu archive but as Linaro’s needs are at times, different, being able to diverge without affecting Ubuntu itself is important. This effort is being lead by James Westby.

Tuesday morning finished with Paul Larson’s testsuite and profiler presentation. Closely related to Zygmunt’s dashboard effort, Paul is working on combing the various testsuites and profiles into one big validation effort.

At this point the guys from Code Sourcery joined us to discuss toolchain’s and related tools. Will Deacon, Matthias Klose and David Rusling joined the conversation and many points were ironed out. Linaro will benefit from the great work Code Sourcery do around GCC and related tools, producing what we think will be the industry standard tool chain for ARM based devices. A lofty goal but everyone behind the effort is confident.

Talk on tools and toolchains continued to be the focus for the rest of Tuesday and Wednesday. Code Sourcery went away with a pile of work items and the Linaro members todo lists were equally full. There was just enough time Wednesday to sit down with Dave Rusling to formally sign off the technical requirements.

As one Linux Weekly News author put it,

the list of planned achievements for the five months before the release is quite ambitious … even completing a big chunk of it would be quite an accomplishment

We believe we can do it and with the great individuals such as those at the sprint this week, the 10.11 release will be something special.

“Any intelligent fool can make things bigger and more complex… It takes a touch of genius – and a lot of courage to move in the opposite direction”.
- Albert Einstein

A little history

Perhaps a little known fact is that well over 15 billion chips have been shipped using ARM’s technologies, 1.3 billion in the last quarter of 2009 alone and as the internet goes truely mobile this is set to sky rocket. ARM have sold over 600 processor licenses to more than 200 companies and 12 of the top 20 semiconductor companies use their technologies. ARM based devices really are everywhere. In contrast, Intel have shipped just 3.3 billion to date. Renowned for their low power consumption and high performance, ARM based devices really do seem to be the holy grail of computing but why is it that Intel continues to dominates the desktop, laptop, netbook and server market?

Fragmentation

Developing for ARM devices isn’t without its problems. There are a lot of companies all working independently on producing their own products, often duplicating effort. Kernels, boot loaders, and to a lesser extent middleware are being worked on in isolation with little in the way of standards and a common direction. This is scary for those who are used to working in the Intel world where one kernel and one boot loader will pretty much work on all compatible devices. To really push ARM devices into the standard spaces Intel currently enjoy’s, something needs to be done.

Ubuntu’s Linux on ARM initative

Canonical, creators of the renowned Ubuntu distribution, and ARM saw the need to rally around an effort to produce a modern, full-featured Linux distribution tailored for ARM devices. Together, starting in 2008, Canonical and ARM took on the task of bring Ubuntu to ARM platforms. Release 9.04, codenamed “Jaunty Jackalope” was the first Ubuntu release supporting Freescales iMX51 and Marvell’s Dove platforms. This distribution was further refined in the 9.10 Karmic Koala release and 10.04, Lucid Lynx even added support for a third ARM platform, TI’s Beagle Board. With a completely redesigned user interface, a web based office solution and many more improvements, Lucid is a magnificent release; however there is still a need for a more consolidated effort. This is where Linaro fits into the story.

What is Linaro?

Linaro is an initiative undertaken by ARM, Canonical and partners with the task of improving the state of the whole Linux on ARM ecosphere. It brings together the vast talents of the open source community and ARM’s wealth of experience in the electronics industry to work on key and game-changing projects. It will work in the various upstreams where possible and provide engineering, technical and guidance support for a wide and diverse set of problem area’s. Linaro will not just help other projects, there will also be a clear set of deliverables which will culminate Linaro’s efforts into regular 6 monthly engineering releases, starting this November.

Release Objectives

So this is the area which gets me the most excited. As the release manager for Linaro I am responsible for making sure each and every release captures exactly what the essence of Linaro is, consolidation, collaboration, improvement and robustness. Each of these qualities are goals for the Linaro releases and to that end our first release, 10.11, will contain all of these in abundance.

Building upon the already successful Linux on ARM effort, Linaro will utilize Canonical’s Launchpad framework including bug management, code hosting and blueprints.

Just some of the high-level highlights are below. I’ll leave the other Linaro team members to blog the details about their teams efforts later.

1. Help standardize the industry on common kernel versions and features.
2. Improve debugging and performance analysis at the kernel level.
3. Bring power management and performance improvements including boot speed reduction.
4. Promote and implement device-tree’s on ARM hardware.
5. Provide test ‘heads’, whole vertical software stacks and distributions, to show what can be done on top of Linaro.
6. Explore integrating telephony right into the distribution.
7. Improve the state of graphics acceleration.
8. Provide QA and validation harnesses to ensure anything built with Linaro is of the highest standard.
9. Offer performance analysis and suggestions on how to improve.
10. Supply a whole host of development, archive and image management, and distribution creation and customization tools.

And much, much more. A full list of blueprints which capture what we are trying to achieve can be found on the Linaro wiki.

Get Involved

For more information on the Linaro initiative please see the official website and the wiki. We have mailing lists and a Launchpad project page and I invite you all to come and participate in this exciting and ground-breaking venture.

What next?

So whats next? Well, the famous words of Albert Einstein that opened this post are most relevant here. We are the Genius’s trying to make ARM development both easier and simpler. So with courage and a bit of luck, Linaro will succeed in bring Linux and ARM to a whole new level.

A new user interface

One of the most obvious changes is the user interface. The ARM version of Ubuntu’s previous release, Karmic Koala, booted to the default Ubuntu desktop. For some this was fine but typically today’s ARM devices tend to be different. At present, they tend to have smaller screens, less resources and little in the way of graphics acceleration. To overcome some of these limitations in the x86 netbook world, the netbook-launcher user interface was created. Based on Clutter, netbook-launcher could not run on the ARM devices Ubuntu was targeting due to a lack of 3D acceleration. Enter netbook-launcher-efl, a 2D version of the x86 netbook interface written using EFL packages.

Read more about the 2D EFL based launcher.

Faster Live CD boots

Booting a Live CD is something that most new Ubuntu users do (and many existing users too). Its often their first experience of an Ubuntu release and should give a good impression. Well, on some ARM hardware, booting this Live CD image took over 3 minutes, not exactly the impression we would hope. So investigations happened into what was causing this slowness. In the end the final boot time was reduced by around 35% on all Ubuntu Live images, not just ARM ones.

Read more about the Live CD boot time improvements.

Web Office and Web Mail integration

Open Office on a resource limited platform isn’t the greatest experience and to make matters worse, on the ARM architecture there are issues building it correctly. A new way of viewing, editing and saving office documents was needed and for the Lucid cycle a web-based solution was integrated into the desktop called webservice-office-zoho.

Read more about the web office integration.

Similarly, Evolution could be considered too heavy-weight for ARM device needs. A solution was implemented to enable integration with several online mail providers.

Read more about the web mail integration.

Optimized Tool Chain Defaults

This release includes a complete archive rebuild using more modern tool chain defaults that the latest ARM hardware can take advantage of. As of Lucid Lynx, packages are built using Thumb-2 to reduce code size and improve performance, NEON for accelerate multimedia and signal processing, and are optimized for ARMv7A based chips. Although this means that some older hardware will not work with the latest Ubuntu release it does mean that the images perform much better on modern hardware.

Other Improvements

Much bug fixing went on this cycle. The fail to build list (FTBFS), a list of packages that fail to build on a given architecture, was a focal point of activity. For the first time ever, the number of packages that failed to build on ARM from the main archive was zero (apart from libx86 which refuses to leave the build queue for ARM due to a bug), a great achievement.

The Chromium browser now works on ARM, rootstock, the tool to build ARM rootfs tarballs gained a gui frontend, we added support for the very popular OMAP platform (beagle board) and many small improvements were implemented, making this the best Ubuntu ARM release ever.

We here at Canonical are very proud of the Lucid Lynx on ARM and are extremely excited at what future releases will bring.

Desktop Integration with the cloud is hot news. Ubuntu One is a great example of this. Currently Ubuntu One integrates file storage, contacts and notes sync, and now you can even buy music from the online store, delivered straight to the Rythmbox media player. But for some devices, integration with the cloud isn’t just a nice feature, it completely changes the user experience (UX). Take for instance a low powered, possibly mobile/embedded system with limited processing power and memory. A cloud based service for these devices could allow resource intensive tasks to be offloaded to an online server somewhere, greatly improving the UX. One set of tasks that are used often but can put a strain on resources are related to office document editing.

Ubuntu’s Current Offerings

The standard Ubuntu image currently contains the Open Office suite. For those that do not know, Open Office is, from the website:

“OpenOffice.org 3 is the leading open-source office software suite for word processing, spreadsheets, presentations, graphics, databases and more. It is available in many languages and works on all common computers.”

Open Office also has around 9 million lines of code spread over 30 thousand files, in short, its a large project. Running this on a device with limited resources is a recipe for disaster. Of course there are alternatives, Abiword and Gnumeric are examples of two applications that replicate functionality found in Open Office’s Writer and Calc respectively. There are also online services such as Google Docs and Zoho but neither of these are tightly integrated with the desktop, until now. Enter webservice-office-zoho.

Why Zoho?

First off, why Zoho and not, for instance, Google Docs. Both services offer great functionality and are very competitive but the ultimate decision came down to which suited our use case the most. What we wanted was for a user to double click on a document which would then seamlessly open ready for editing. From there the user would edit, read, and maybe even save it back to the local device. No fuss, no logging in, no other requirements, just open and get on with it. Similary, when a user launches the application on its own, we wanted the correct type of service to open ready for the user to concentrate on their document. The service that allowed us to do this was Zoho. Zoho allows the user to do all of this without ever registering or logging in. Of course you get online storage with Zoho if you do register, but if you choose not to you can still get a full featured experience.

Integrating Zoho with the Desktop

So what do you get with this new integration. Well as alluded to before you can:

• Open, read, edit and save email document attachments.
• Open, read, edit and save local documents.
• Open, read, edit and save remote documents linked to with a url.
• Launch the required application, Writer, Show or Sheet (Word Processor, Presentations, Spreadsheets) which will present the user with an empty document of that type ready to edit.
• More functionality to come in Maverick Meerkat.

webservice-office-zoho shown here installed along side Open Office

The Future

This functionality is currently only available as default on Ubuntu’s ARM images, typically where limited hardware resources are more commonly found. But that’s not to say webservice-office-zoho can’t be used on any other Ubuntu install. As the package is in the main Ubuntu repository, a simple:

sudo apt-get install webservice-office-zoho

will install it on your Lucid based machine or if you are feeling brave, checkout the latest bazaar branch with:

bzr branch lp:webservice-office-zoho

There are lots of things planned for the future of webservice-office-zoho. If you have comments, idea’s or just want to rant, come along to the web integration UDS session this May, either in person or via online methods or just leave your thoughts here.

Vendors are trying to work around it, especially as there is the expectation of a ramp-up in the availability of ARM based hardware. Super long-life netbooks, low powered touch based computers, and even a flurry of smaller embedded devices are forecast to hit the market this year, many of which will be based on the Linux operating system. Ubuntu would be a great match for this.

Ubuntu and ARM

Ubuntu runs very well on some ARM based platforms and there is a sustained effort to make it work more ubiquitously across many more. To that end our goal is to have Ubuntu running on any ARM based device (as long as there is hardware available). A lofty goal but one which we would like to see happen.

So what can we do about the 3D graphics licensing issue? Legally not very much. The companies that own the IP (Intellectual Property) rights to these drivers often want large licensing fees for their technology. This is a model for single product lines (take the Nokia N900 for instance) but for Ubuntu where we are targeting a more broad approach, this isn’t ideal.

So when you buy your new, ARM based netbook that has an obscene amount of battery life and you just want to install the 3D clutter based, wonderfully rich UI that Ubuntu Netbook Edition offers, what do you do?

Well Ubuntu recognizes this problem and as part of the Lucid Lynx release there is an effort to bring a similarly wonderfully rich UI to non-3D-accelerated hardware.

The new 2D EFL based Launcher

Above you can see the default UI for Ubuntu’s ARM based releases starting from Lucid (10.04). It’s a direct clone of the UI found in the 9.10 Karmic release on i386 although this one is based on EFL (Enlightenment Foundation Libraries) meaning that its fast on non-accelerated platforms. If there is 3D hardware available it can use that but it works perfectly fine without.

Another great thing about the 2D launcher is that isn’t not restricted to ARM hardware only, in fact if you have Lucid installed now, getting the launcher couldn’t be simpler. At the command prompt just type the following (make sure you have the universe repository enabled):

sudo apt-get install netbook-launcher-efl

and voila, your UI switches to the new launcher. Of course a simple:

sudo apt-get remove netbook-launcher-efl

will remove it if you decide its not what you want.

Beyond Netbook Launcher

Another of the great things about this launcher, as apposed to the 3D launcher shipped with Karmic, is that its extremely theme-able. The theme file is contained in:

/usr/share/netbook-launcher-efl/data/themes/default.edj

Theme files use the edje declarative layout format. By changing this file you can completely change the way the UI looks. For example, see the alternate UI screenshot above, both are based on the same code, the only difference is that they have a different theme file.

So if you have ARM based hardware but no 3D acceleration, fear not, you can get the same great user experience that your i386 cousins have in Ubuntu Netbook Remix.

How do you boot a live cd session?

The first thing to do was find out why it was slow. There are a few ways to do this but I chose to first use simple time-stamping methods and afterwards, the much prettier bootchart package.

A bit of background on how the live cd session is booted. There are two broad steps in the process of booting a live cd, the first is setting up the environment ready for the session, and the second, you guessed it, is actually booting into the session. Initial hunches were that the first step, setting up the session, was the major cause of slowness so investigations started there.

Casper

Setting up the session is the responsibility of a project called casper. Casper is a set of scripts that are run on boot to do such things as unpack the initial filesystem, add a dummy user, setup languages and keyboard layouts and so forth. Its mainly written in perl.

The time-stamping stage of investigates confirmed that casper was indeed slower than it should be. I’ll skip ahead to the bootchart part of the story as thats much more visually interesting.

Casper before

Above you can see the casper section of the live cd boot process. I’ve highlighted the bits that immediately stand out. The overall boot was 3 minutes and 15 seconds with casper responsible for around 2 minutes of that.

The highlighted bits seemed to have one thing in common, they all interacted with the debconf database. From the debconf programmers tutorial document:

“debconf is a backend database, with a frontend that talks to it and presents an interface to the user. There can be many different types of frontends, from plain text to a web frontend. The frontend also talks to a special config script in the control section of a debian package, and it can talk to postinst scripts and other scripts as well, all using a special protocol. These scripts tell the frontend what values they need from the database, and the frontend asks the user questions to get those values if they aren’t set.”

So communicating with the debconf database was slowing the boot. Initiating the call, sending the data and receiving a response, was taking up to 4 seconds at a time and when there are many of these calls, they all soon add up.

After many a head-scratching moment it was decided that the best way to solve this would be to initiate the communication once and keep it open so that when debconf was needed, the overhead in setting it up was removed. The implementation details are all in the code history but the results are much easier to show.

Casper after

With a couple of other tweaks besides the debconf one, the boot is now down to 1 minute 53 seconds and casper takes just under 50 seconds of that. There is more room for improvement, pre-generating a default locale (although which locale that would be is a tough choice), pre-generating fonts and looking into SSL, but for now, this is a big win.

The bootcharts above are for the ARM based iMX51 device made by Freescale which is only just beginning to proliferate onto the market. Intel/AMD based machines show an equivalent speed-up.

Conclusion

So there you have it. The next time you boot into a Lucid live cd session it shouldn’t take quite so long, and now you know why.

So its that time of year again, a great Ubuntu release was made late last year and the team is frantically trying to make the next release even better. So to facilitate this the Ubuntu platform team got together for a sprint in the most awesome city of Portland, Oregon last week. Whoever designed the city must be commended, its easy to explore, well laid out and you never get the sense that you are lost, much kudo’s to them.

Anyway, back to team business. The mobile team managed to get a metric tonne of work done (possibly more) in the 5 short days. To date we now have a new default UNR styled UI, 33% faster-to-boot live-cd images (thats across all images, not just ARM), mature and optimized (can you say Thumb2+NEON) images, the Chrome web browser on ARM, greater integration of testing and many, many, bugs squashed. All this and we are only half-way through the cycle.

ARM hardware is only just proliferating onto the market but when it becomes mainstream, we hope our hard work will make Ubuntu the default operating system. The team is working hard to make Lucid the best ever ARM optimized Linux release and we invite you, if you have hardware, to use, test, and contribute to make Linux the choice for ARM hardware.

Now I’m a big media fiend, just ask my wife. We now don’t have a garage because I went and converted it into a home cinema room (8 foot screen, HD projector, 500+ films archived, DVD+Blueray+HD-DVD sources …). I hack on the Entertainer Media Center project (although its a little stale at the moment, effort really appreciated) and I’m a former user of XBMC on the first generation XBOX’s from Microsoft and now uPNP of the XBOX 360. So when someone says to me they have a £150 tiny and silent PC, running Linux and decoding HD material, I got a little credit card jumpy.

A couple of days later the Revo arrived. I was a little in two minds on whether this was a good purchase or not. I count at least 23 computers in my house so another was probably not a good idea but none-the-less, I justified it as a replacement for the excellent but slow Viglen MPC-L which, coupled with many USB disks, was a great file server. The first thing I did was install Ubuntu Karmic Koala. Out of the box it worked great (no surprise there ). Install the Nvidia drivers, tweak the screen resolution for my plasma TV, install XBMC and away we go. One tweak every media concerned individual must do on this device is to disable compiz. For some reason, on full screen playback, compiz causes video tearing. It took me a little while to figure this out and I don’t mind admitting that I was initially disappointed to see video playback on this machine (when compiz was enabled) but my god did things change.

I enabled my old Windows Media Center remote, added all my video sources to XBMC, tweaked the interface a little and now, all I can say is WOW. The Revo+XBMC combo ranks up there with the best of my purchases. The interface is wonderful; the little things like showing you the actual time a video you are watching is going to end, the UI which puts every set-top-box maker to shame and the shear extensibility of the thing is immense. I won’t even mention that this is running Ubuntu, doing my work-related backups, downloading (legal) torrents, running scripts, mail, and a whole host of other services, and drawing a trickle of power.

If your considering replacing a uPNP enabled XBOX (which I did) or PS3, getting rid of your aging server or just wanting a great media experience, I whole-heartedly recommend this combination.

Well in the interests of fairness one Canonical employee decided to send the reporter a netbook with a copy of Karmic (the next Ubuntu release) on it.

Here is the reporters response.

I think its pretty far off the mark. Comments such as

“But, even after some help from a Canonical advisor who came and installed a few add-ons such as Flash, I struggled to work out how I would organise photos, music and video with this system.”

are a little off the mark. Linux (not just Ubuntu) has a plethora of applications that can manage photos (oh how I love f-spot), music and videos so I’m not sure where the confusion comes from. I attach my USB camera or phone and I get offered the chance to import to f-spot, I open a video file and equally its easy to play it. As for flash, go to a flash based site and you get presented with an option to install not just flash from Adobe but other free alternatives.

Not wanting to get personal, as the reporter Rory Cellan-Jones may not be up-to-date with his market analysis but comments like:

“Risking another pasting from its supporters, I’ll predict that Ubuntu will remain a very niche product – but it’s Google’s Android which could bring open-source to the mass consumer market.”

show a little ignorance. Open source on the cell phone is a little different from the desktop. Android on anything but a cell phone platform is like trying to make a nun get drunk with beer whilst smoking crack (recent small bit of ammo).

The phone war will be fought with Nokia’s Maemo (but not the current generation) and Google, the desktop (which he was testing) will be fought with Google (Chrome OS), Windows 7 and some Linux flavour.

I look forward to an unbiased, educated analysis of the technical market by the main-stream media but I don’t hold my breath.

Installing a Virtual Machine

My virtual machine software of choice on Ubuntu is the excellent VirtualBox. The easiest way to get this on Ubuntu is to apt-get install it on the command line. Open up a terminal (Applications->Accessories->Terminal) and enter the following at the prompt.

Once installed you will have an option under Applications->Accessories->VirtualBox OSE. Click this and you should see the VirtualBox application in front of you. Select the “New” icon and follow the prompts. You need to name your image something that you will remember, maybe “Ubuntu NetBook Remix”, and make sure the operating system is set to Linux and the version, Ubuntu. Give the image 512mb of ram and at least 2GB of hard drive space (it can survive on less but I always like to leave some room).

Getting the image

If your helping test UNR you really want to be trying the ‘dailies’. These are complete UNR images that are built every day. The place to get them is at cdimage.ubuntu.com, specifically at

http://cdimage.ubuntu.com/ubuntu-netbook-remix/daily-live/

The “Current” link always points to the most recent build. Select the iso image that looks something like below (karmic release in this example).

karmic-netbook-remix-i386.iso

Setting up the Virtual Machine

Once this is download you want to make sure your virtual machine image will boot into UNR when it first runs. To do this select the “Settings” icon from the VirtualBox screen (first make sure you have selected your image in the left-hand column).

What you are presented with now is a list of options for your virtual machine image. The one we are interested in is CD/DVD-ROM. Select this option. Below you can see an example of what it should look like once configured correctly.

Use the folder icon next to the ISO setting to browse for the image you downloaded.

Press OK then hit “Start” whilst your image is selected to run it.

Ubuntu Netbook Remix

If everything has been done correctly you should be presented with the boot screen from Ubuntu asking whether you want to try it, install, e.t.c (after you have selected your language). Go ahead and install it to your virtual hard drive.

Once installed remember to disconnect the CDROM (Devices->Unmount CDROM from the launch window) to ensure that you don’t boot up the install .iso again.

If everything was installed you should be presented with something like this.

If you get a message telling you to disable KVM in the kernel, a quick,

lsmod | grep kvm
sudo rmmod kvm-intel
sudo rmmod kvm

should fix it.

The lsmod finds any running modules that are related to KVM (which in my case are kvm-intel and kvm). The two commands afterwards remove them from the running kernel which enables VirtualBox to run.

Bugs?

If you find any bugs with Ubuntu Netbook Remix then be sure to either file them at the Launchpad site (Netbook Remix on Launchpad) or discuss them with the developers on irc at irc.freenode.net channel #ubuntu-mobile.

Most of all, have fun.

Additions

It was pointed out to me by Paul Larson that the resolution that VirtualBox gives you for UNR by default isn’t what is traditionally found of netbooks. The more widespread 1024×576 pixels is what most machines have.

To enable this in your virtual machine, head on over to Paul’s blog and read how to do it. What you should have at the end is something more like this.

Please note that you need to have the vbox additions installed to do this.