Pierre-Luc Beaudoin

10 days before Maemo Summit 2008

As it has already been pointed out, there are only 10 days left before the opening of Maemo Summit 2008. I’ll be attending this first summit with Marco Barisione, one of my colleagues from Collabora. We’ll both be presenting.

On Saturday September 20th at 15:30 for half an hour, Marco will introduce you to Telepathy, the framework used in the maemo platform for instant messaging and audio/video conferencing.

On Saturday September 20th at 16:30 for half an hour, I will introduce you to WebKit, the open source web engine. I’ll be explaining part of the internals and demonstrate how to use it in your Gtk or Qt application today (therefore this talk is oriented towards Gtk+ and Qt, which are toolkits  you can use to build applications for the maemo platform).

The summit is right after OSiM World, at headquarter of c-base in Berlin, Germany.  I hope to see you in great numbers at both our talks!

Schedule subject to changes, see the official wiki page for times and details.

NPAPI plugins are supported

This Arstechnica article is quite interesting…

Also, NPAPI support is already in the Gecko web rendering engine.

I’d like to correct a perception here: NPAPI plugins are supported in QtWebKit (current SVN and future releases) and WebKitGtk.  In fact, Marc Ordinas i Llopis made it work months ago!

Making cool things with WebKit

Everybody have heard that WebKit is the new cool kid.  Here is one reason: it is rather easy to hack on.   My colleague at Collabora, Siraj Razick, mixed up QWebView and QGraphicsItem to allow developers to put the Web inside a QGraphicsScene.  As he demonstrates, you can do crazy graphics with that.

From there, I went to look at how we can implement the same behaviour we have in one of the popular mobile browser these days.  Here’s the result: stefani.ogg (Stefani is the name of our hacking app).

In this video, you can see that each time I click on a paragraph, its content is zoomed to fill the screen.  If you click again, you get back to the full page view.  At all time the page is live and active (it isn’t a static image), although that can prove to be an issue (like mistakenly clicking on a link while in full page view).

You can try it if you want, just keep in mind that this is a quick prototype and it doesn’t really support scrolling yet.  The code is here: http://git.collabora.co.uk/?p=user/pierlux/stefani.git;a=shortlog;h=refs/heads/pierlux-zoom 

Now more details on how this works: on double click, I do a hitTestContent() at that position.  With a patch we wrote, you can access the HitTest’s bounding box.  That is how I determine on what to zoom.  After that, it is simple mathematics, signals and QTimeLines.

Granted this method is rudimentary and does not always gives the best result, it works with patched current available API.  A more elegant solution (which we may write in the future), would be to use a DOM API and access the node that was clicked on.  From there, you could go up the DOM tree and see if a particular div would be more interesting to zoom on  (ie, if a <p> tag is contained in a <div>, zoom on that <div> might also contains the title of the paragraph).  But first, we have to write that Qt DOM API :).

The cost of a bug fix

Every fix doesn’t call for a blog post, but this one deserves it.  It all started when Jonathon Jongsma found a way to make text disappear in QtWebKit on May 27th.  So he raised a bug.  He and I started working on fixing it.  We rapidly found that WebKitGtk was also affected, but it was unreproducible on the Mac port.

We dove into the code: “grep selection”, GraphicsContext::drawText(), Font::drawText()… but nothing was really different (there) in the Qt or Gtk port which could explain why the text wasn’t being redrawn when changing the selection.

That’s when I discovered git bisect.  Since we had established that the bug wasn’t there when QtWebKit was snapshot for Qt 4.4.0, I had a good place to start.  So after recompiling QtWebKit some 15 times (yes, it took around 3 work days!), it pointed me to this changeset.  Lucky for us, it was related to the bug (text rendering).

After some digging into the patch, I contacted the author, Dan Bernstein at Apple, and we looked at it together.  In little time, he was able to find how to reproduce it on the Mac too.  This was now a WebKit wide bug!  Some back traces later and some trials: we came up with this fix. Pretty simple, isn’t it?  barely 16 chars.  Yet, these 16 chars cost around 1200$* in direct labour time and 3 engineers were involved.

Some will say this could have been prevented with proper tests.  It happens that it was a special case on the Mac, but all other ports always went through it.  Dan now added a pixel test.

The morals of the story are:

• bug fixing is costly (haven’t we heard that in school?)
• you never know when someone will hunt you back about your patch
• git is a cool beast (in fact, it just convinced me to use it)

One question lasts: how come it took over a month and a half before someone found it?

* This number is based on market mean hourly rate since exact rates are unknown

Side note on the WebKit party

It was really cool to get to San Francisco and finally meet IRL other WebKit devs.   Kudos for the event!

Permettre l’usage des méthodes de saisie

Je suis surpris de ne pas avoir rencontré ce bogue moi-même, après tout j’utilise un clavier Canadien Français. Avant hier soir, il était impossible d’utiliser les touches mortes (souvent utilisées pour les lettres accentuées) avec WebKit/Gtk+. Or, une simple rustine de correction de 20 lignes a suffit pour compléter le travail déjà entreprit par Alp Toker et Christian Dywan pour permettre l’usage des méthodes de saisie.

Je ne savais pas que l’utilisation des touches mortes dépendait d’une méthode de saisie. Il m’a donc fallu faire quelques recherches sur le sujet en premier lieu. La documentation sur GtkIMContext est un peu limitée et je n’ai pas trouvé de documentation faisant le survol de cette technologie.

Bref, la bonne nouvelle c’est que non seulement on peut maintenant entrer tous les caractères accentués dans un formulaire Web, mais cette rustine permet également aux autres méthodes de saisie de fonctionner! On peut donc maintenant saisir de l’arabe, du thai et plus. Avec un correctif attendu pour les polices asiatiques, WebKit/Gtk+ est en voie de devenir une platforme de rendu de page web de classe internationale.

Mise-à-jour: Il semble encore y avoir des petites retouches à apporter pour le Japonais, par exemple. Mais bon, il faut quelqu’un qui comprend cette langue pour trouver ce bogue!

Ogg Vorbis/Theora Language Removed From HTML5 Spec

Sad news for HTML5, while Ogg Vorgis/Theora were not necessary to make HTML5 work, a standard codec is needed to make some part of HTML5 usefull - the part I worked on for WebKit/Gtk+. This point has been repeated over and over: all browsers implemented HTML5 needs to have a common codec.

While there is no default image format in the HTML* spec, we were lucky nobody ever used their patents against JPEG. In fact, it is most probably because of said patents that Apple and Nokia were reluctant to include Vorbis/Theora on their OS or their phones.

Someone has to fix that patent system!

• Slashdot - Ogg Vorbis/Theora Language Removed From HTML5 Spec

HTML5 media tags in WebKit/GTK+

I’ve been working on bringing support for media tags in WebKit/Gtk+ in the past weeks. Based on Antti Koivisto’s work on the general WebKit support for the tags and his QuickTime implementation, I was able to provide the basics for this to work.

Sure there are sharp corners right now, it’s a good start. While I did the backend part, connecting WebKit and GStreamer, Alp Toker did a GStreamer sink for Cairo that completes the work. You can see the screenshot on his blog.

Media tags will enable web developer to integrate video in a page as easily as it is for an image nowadays. You could eventually implement a complete media player within a HTML file. With all the major browsers publicly announcing their progress on the matter (Safari, WebKit/Gtk, Firefox, Opera), we can sure hope we’ll see less flash and more pure video in the next years.