What we’ve been doing for two years, is porting Chromium to Wayland!

Juan José Sánchez loves Wayland.

Igalia is an open source consultancy that has been fighting the good fight for nigh-on 16 years. It’s contributed extensively to (and maintained) the WebKitGTK project and does all manner of other impressive work in areas from multimedia to virtualisation.

Last month Igalia co-founder Alberto Garcia gave us the lowdown on some of his work with

Qemu, as well us making us nostalgic for ZX Spectrum games. This month, his colleague and fellow co-founder Juan José Sánchez takes the mic ( you mean “affordable smartphone”– Ed) to tell us about the gains Wayland is making in the embedded sphere, and the ongoing effort to support it in the open source

Chromium browser, as well as its own embedded effort, WPE. Juan has a Ph.D. in computer science, understands functional programming and is an all-round nice guy. We caught up with him at the Linux Foundation’s Open Source Summit in Prague last October.

Linux Format: It’s become tradition for me to ask about how people got into computers and Linux. How did your journey go?

Juan José Sánchez: When I was eight my father brought our first home computer, a Philips NMS. All you could do on it was program in BASIC and look around inside directories. So I started out with that. I programmed in BASIC for years, took a couple of courses and read a few books. Then in high school I was programming for my father’s small business, and wrote some management software.

I studied computer science at the University of A Coruña, where Alberto also studied. There was quite a strong Linux User Group there and we got involved. Actually, that’s where Igalia came from: people connected to that User Group. When we finished our studies we thought it was a good idea to try and create this company to try and work on open source and free software. There isn’t really much industry in the region of Gallicia where we’re from, so it was a nice opportunity to be able to work remotely from there. Since then we’ve been doing… well, all kinds of free software things. I don’t think we could be this kind of company without Linux or open source – we’d be much more irrelevant in a proprietary world.

LXF: What’s your Linux distro of choice?

JJS: I use Fedora nowadays. I used Ubuntu for a long time, and I’m thinking of switching to Debian. I like changing. But also, from a practical point of view, a lot of our work is connected with Gnome, and Fedora provides a regularly updated Gnome desktop. I’ve always liked the Debian project too – in fact, I started using it at the very beginning.

LXF: I use Arch at home and work, but this laptop usually gets a different distribution put on it every time I go on a trip. It sort of adds to the journeying aspect: you go to a different place, you have a different distro to fight with.

JJS: Cool. I like Arch, but lately I don’t have much free time for playing with distros. I guess I’ve became a bit more practical and pragmatic. But still, Fedora and Debian are nice. In some ways it feels like there are fewer distros than there used to be. Things have converged. In Spain there was a period where lots of local distros were being created. I remember the Brazilian distro Conectiva Linux, that joined with Mandrake, which was French. I remember trying them back in the day – they were interesting!

LXF: And Mandriva has now been forked in the form of Mageia and OpenMandriva Lx. Tell us about some of the changes you’ve seen in companies’ attitudes towards open source while you’ve worked at Igalia

JJS: We’ve been around 15 years, and in that time we’ve seen quite a change. In the beginning we were trying to convince companies to use open source software and Linux, and why it was interesting. We spent a lot of time explaining the details to prospective clients. A few years later it became easier to convince people about the merits of FOSS, and it became much more widely accepted. And now, well, the clients basically come to us. In some cases they go straight to the repositories to see who’s committing and get in touch with that developer directly.

So yes, they’re much more interested now, and it’s much easier for us to establish business contacts. Especially the stuff we’re doing with web browsers. Today people want browsers running HTML5 applications in vending machines, speakers, elevators and so on. This wasn’t possible three years ago because there were a lot of performance issues and security concerns. But now performance is comparable to native for many things, and this opens a lot of exciting possibilities for us.

LXF: In the previous interview Alberto told us that most of your engineers are involved in web browser work. Can you give some examples of that work, and the types of companies that you’re doing it for? JJS: Our browser work is divided into three teams. We have a Webkit team, a Chromium team and a Web Platform team. The latter does a lot of standards work: things like CSS Grid Layout, which is a W3C standard for arranging html on webpages (read more at https://alistapart.com/article/the-story-ofcss-grid-from-its-creators). We did the implementation from scratch in both WebKit and Chromium. We work with lots of different kinds of customers, too. Often it’s device

On the shrinking FOSS world “It feels like there are fewer distros than there used to be. Things have converged”

on alternatives to android “I still like small projects that try and rekindle the Linux on mobile dream”

manufacturers or hardware vendors that want to improve or optimise FOSS for their needs. Some clients are stranger. For example, we did some work on CSS layouts that was sponsored by Bloomberg, for some strange reason.

LXF: That is strange

JJS: Well, they have all these displays that show live market data and so forth. They use

Chromium for this, and it’s quite an extreme-use case because there’s a lot of JavaScript, lots of objects, lots of graphs, and so things could get a bit slow. So they figured that the new Grid Layout standard could help and they paid us to push this standard. Basically we were doing R&D for them. It’s a nice collaboration for us: there are no strict deadlines, things are very free ( jealous grumbling noises – Ed).

We also do other work with lots of other companies. Google is sponsoring us to work with WebKit, which is also very surprising, but it’s all to do with Google AMP. There are some compatibility issues there with WebKit and iOS that it wanted us to fix. So we’re working in WebKit with Google to improve Safari. An odd collaboration indeed!

In terms of services offered, Igalia provide training, consultancy – meaning analysing different technological options, helping people to select and define their architectures – but most of all, almost every project in fact, is about development with some upstream integration. So in some sense we become the link between companies and the open source community.

LXF: Your booth demonstrated some pretty slick browser animations running on a Raspberry Pi. Tell us about the underlying tech there. JJS: We have been working on embedded browsers for some time, almost ten years. We were using all kinds of different solutions. There was QtWebKit, WebKitGTK (which we maintain), and some people were using Chromium in the past couple of years. But we kept noticing that there was always something missing. We needed a solution that had been designed for embedded usage from the outset. So it needed to be independent of any toolkit and it needed to be flexible so that it could be ported to different architectures.

We knew for several years that we should do this, but we never found the time. Then in 2014 we started the WPE (Webkit Port for Embedded) project. Initially it was more of an experiment, but over the past two years we’ve been working very hard on it and that work has been mainly sponsored by a company called Metrologic. It’s developing set top boxes as part of the RDK construction. By late 2016, we had essentially a useable product, and in May 2017 we pushed it upstream. Now it’s starting to fit with other people’s requirements and seeing wider use.

We started with some things from the WebKitGTK port, which we knew very well and was very stable, and we removed the toolkit part. We thought about using Wayland as the graphics back-end, because it was fast and becoming very popular with some of our customers. Then we tried to keep the dependencies minimal, so that everything was small. The idea was to have something that was small on disk, small in memory and as fast as possible given those limitations. So, something that can work on the Raspberry Pi 1, or even less-powerful devices: we’ve experimented with the Pi Zero, Freescale devices and the i.MX 6 as reference/target hardware.

We started optimising the rendering layer and the video recording layer using Gstreamer, so that everything was done using hardware acceleration. This is still an ongoing project, but it’s already being used on a considerable scale. The two main companies involved are Comcast and Liberty Global, and they’re using it in a huge number of set-top boxes. I think the last figures I saw were that about ten million boxes running an instance of WPE on a classical Linux stack. WPE has a flexible architecture. It can be used with a number of back-ends besides Wayland. In some cases we use libWPE with Mesa, and there are back-ends that use proprietary drivers, for example the Raspberry Pi. So you can select the right back-end for your hardware and the type of graphics you’re doing, and the rest of the WPE code is re-used. LXF: I think a lot of our readers will be surprised that Wayland is something more than an experimental technology that does weird things to their Gnome sessions. It’s edifying to know that it’s already being used, and on such a considerable scale. JJS: We get a lot of that, too. It’s interesting! In the early days WPE was nicknamed WebKit for Wayland, and peoples’ immediate reactions were “This doesn’t make sense, this isn’t a mature technology, why are we using it? It’s still going to be experimental for years… X11 will be around forever”. But this isn’t what we’re seeing now. Some companies are really beginning to embrace it. Now that Fedora has switched [to Wayland by default] it’s easier to convince people that it’s a viable technology. Up until very recently, most people thought it wasn’t ready. LXF: I guess much of the problem is that desktop Linux applications still need to be ported to Wayland, rather than any shortcoming of the technology itself. JJS: It’s a lot of work. One very good example is Chromium – we have a demonstration of it in our booth actually. But today, with Fedora, if you want to use Chromium on Wayland (in the default configuration), it uses Xwayland, which is a kind of wrapper [that translates X11 calls to Wayland ones]. It’s pretty slow. I mean, if you have a fast-enough system and aren’t doing anything too crazy you probably won’t notice it, but in relative terms it’s very slow.

So what we’re doing – and what we’ve been doing for almost two years – is porting Chromium natively to Wayland. And it’s still not

ready, so yeah, it’s a lot of work. I guess most application developers are daunted at the prospect of having to do this migration. In our case it’s being sponsored by Renesas.

LXF: Renesas is also part of the Civil Infrastructure Project, which I learned about in an earlier talk. It seems to be pretty involved with Open Source in general. Back in 2016 I talked to Dan Cauchy about Automotive Grade Linux (AGL, see LXF218 – Penguins Under the Hood). JJS: Our collaboration with the company is very close to AGL. It’s sponsoring us because it wants Wayland-powered Chromium as part of the AGL platform. It’s going to be using it for In-Vehicle Infotainment (IVI) systems.

LXF: What about Linux on Mobile? That seems like a reasonable use case for embedded browsers. But Ubuntu has given up with it, and lots of other projects have fallen by the wayside.

JJS: There was a huge opportunity, actually with Maemo which started in 2005 – two years before the iPhone. Because of strange decisions within Nokia, which everyone knows about, it ended up failing. But the technology was very good, and I think all the other mobile Linux projects – Ubuntu, Firefox OS and so on – just came a little too late. Android had already dominated the market and offered an easy solution for all the manufacturers. We’ve seen other mobile efforts from other companies because people really want an alternative to Android, but it’s really difficult for them now that Android’s become so established. It’s a pity. I still like all these small projects that appear every now and then, and try and rekindle the Linux on mobile dream.

LXF: PostMarketOS is looking pretty interesting just now. But there’s also the Neo900 project to make a replacement board for the N900…

JJS: Ha! I was using the Nokia N9 until recently. It’s very difficult though, because the standard communication apps aren’t available so you kind of end up isolated from the world.

LXF: I know that feeling. It’s funny – the postmarketOS page lists a number of devices as ‘working’, but this doesn’t mean you can make calls or anything convenient like that. It simply means that you can plug a USB network adaptor in and connect to the Internet.

JJS: It’s fine to use these things on a second phone though, just not as your main device. But in general, the story of open source on mobile is a sad one. Maybe it will change though… That’s just one area, too: people are focussed in other areas. For example, in automotive and even smart TVs, lots of companies there want their own Linux-based OS, they don’t want something like Android. So that brings more diversity and makes things more interesting.

LXF: What about the in-flight entertainment systems on airplanes? Sometimes when these systems stop working ( crash!?–Ed) you can catch a glimpse of some Linux.

JJS: Actually, we went to a conference all about that in 2016. Most companies there weren’t using Linux, but we talked to two companies who were. They were using Clutter-based interfaces, which as you know are native Linux technologies. But in general that industry hasn’t embraced Linux as much as it could. Again, maybe that will change in the future – it’s a slow-moving industry.

LXF: So that was WPE running 60fps video on a Raspberry Pi 2. Nice. How does that compare to Chromium that’s installed on the same hardware?

JJS: There are a couple of WPE demonstrations on the booth, both of them running on Raspberry Pis. One demonstrates YouTube TV and all the standards that we need to support there: EME (Encrypted Media Extensions), MSE (Media Source Extensions), WebM. The other is showing off performance, decoding and rendering full HD streams, applying CSS animations. Of course, you can easily run

Chromium on a Raspberry Pi, but you’d struggle to get this kind of performance. Memory is a big issue here, too – Chromium doesn’t like to be constrained.

LXF: EME being approved by the W3C has generated a fair amount of controversy, but ultimately it allows commercial content creators to feel more at ease sharing their work. Can you tell us about some of the challenges implementing this? JJS: You mean technically or about all the polemic behind the standard?

LXF: Let’s go with the first one.

JJS: Good. We’ve started working on it three years ago because it was a top priority for RDK. The initial version implemented everything we needed to reproduce content. That involved quite a lot of Gstreamer and WebKit changes. It was a lot of work for four people.

Once we got the certification for YouTube TV we started working on version 3. This is more interesting than its predecessors because it’s object oriented, and it has a more elegant promise-based architecture. We’re adding support for lots of different CDMs too: WideVine, ClearKey, PlayReady. Every year Google updates the certification requirements for YouTube. Companies get worried about this because they want to be able to continue using the certification and maintain compatibility. We’re working heavily on that, and if you check out the WebKit repo you’ll see a lot of activity there. It’s challenging work: it requires a lot of media and WebKit knowledge, but we expect to have version 3.0 ready soon.

Also, not really related, but we’re working on WebRTC support for WebKit. Apple finally decided to add it, and for a while there they were ignoring it. Parts of that work are shared and we work on the specifics of WPE ourselves. For that we’re using a library called libwebrtc from Google, it was originally for Chromium, but

Firefox uses it too. Initially we were planning to use OpenWebRTC which is more or less a

GStreamer- based equivalent, but that turned out not to be mature enough. Eventually, we would like to have GStreamer integration, so we will look again at OpenWebRTC in the future. We have a basic working prototype at the moment – you can communicate with audio and video, but a lot of functionality is missing.