When Linux won’t do!

Jonni Bidwell meets Kate Stewart, senior director of Strategic Programs at the Linux Foundation, to talk about Zephyr, licenses, garbage trucks and reindeer.

Jonni Bidwell meets Kate Stewart, senior director of Strategic Programs at the Linux Foundation, to talk about Zephyr, licenses, garbage trucks and reindeer.

BRINGING ZEPHYR TO REINDEER “These tags go on the reindeer so they can be tracked and geosensed. People can see if they’re moving or not, and things like that. The battery has to last a long time.”

Back in LXF247 we interviewed the Zephyr Project’s Thea Aldritch, where we learned Zephyr is a tiny real-time operating system (RTOS) that is destined for great things. Zephyr is not Linux, but that didn’t stop kernel don Greg Kroah-hartman, in our last issue, describing it as one of his favourite Linux Foundation projects.

Kate has had over 30 years’ experience in the software world as a developer and a manager. She’s also a key player in the SPDX project, which aims to sort out code licensing once and for all. She gives us an update on what’s new with the Zephyr project, and how it’s learning from Linux.

Linux Format: Can you tell me about your role at the Linux Foundation?

Kate Stewart: Well I’m the director of the project board, so that means I interface between the board and the technical community. I work on strategy and try and build relationships within various parts of the ecosystem. I help to build the ecosystem and try to make sure that any problems that come up are addressed.

There are a lot of places I’d like Zephyr to go this year. We just formed a relationship with the Eclipse IOT people, because they’ve got communication stacks and protocols, but they needed a good underlying RTOS to power them all – they had an open spot there. So finding where we can be complementary and reinforce other open source projects is something we feel strongly about.

LXF: I met with Thea at the Open Source Summit in 2018, and one of the things that made the Zephyr Project so special was the openness of the community. Can you speak about that?

KS: The community has continued to grow. Right now we have about 550 developers who have committed [code] to the GIT repo. So people seem to enjoy working with us, we’re getting a very positive reaction and response. There’s growing pains right now, a little bit anyway, but that’s what happens when you’re growing.

We’ve got a fair amount of cool things emerging into the repo, some things that we really weren’t expecting. Last year we added 64-bit RISC-V support (http:// bit.ly/lxf260risc), that was contributed by Nicolas Pitre. And then last week the OPENPOWER Foundation (see news LXF255) contributed a port for the 64-bit POWER Architecture. I wasn’t expecting Zephyr to be in the 64-bit space initially. So people are finding it useful for various purposes.

We’ve been keeping the community aspect front and centre. Our Slack channel has well over a thousand contributors in it. And we’ve seen a tremendous number of visitors to our website, and downloads from Github and such.

LXF: I visited your website this morning. KS: Ah so you’re one of that number then. Thanks. We’re actually revising the website in the coming months to make certain areas more approachable. But the community mostly interacts with Slack and in the repo lists and such.

We’re having a hands-on session at this conference actually, so people can play with a Zephyr-powered board and get basic communication sorted. Nordic donated the boards for that.

LXF: I’ve seen some Nordic Semiconductor boards, they’re rather cool. And they have funky names. I think it was the Thingy:91 I was looking at.

KS: I wanted that one too, but for our hands-on we have the Thingy:52. There’s actually lots of really cool products coming out with the 91 soon. There’s one called Anicare (see http://bit.ly/lxf260nordic), which is an ear tag for reindeer and other animals in Scandinavia. These are things that you wouldn’t expect, but because Zephyr’s built and designed for low-power and long-life applications it’s ideal.

These tags go on the reindeer so they can be tracked and geosensed. People can see if they’re moving or not, and things like that. The battery has to last a long time, and it’s much better than the big collars people used to use for tracking. Apparently it makes sense for the hearders, because if anything happens to the animals and they can prove that it was something natural like a wolf attack, then they get the insurance. So that way they can see if it’s worth doing the extra trek.

Another application has put devices into boxes that are welded onto the side of garbage trucks. Why? Well, these trucks are meant to go to landfills. So the device triggers when the driver dumps the cargo, and sends a geolocation signal. This can be of use to catch people that aren’t using landfills – there’s a big problem with illegal

flytipping. So there’s all these really weird, interesting new applications that are coming up with Zephyr, and that part of the community is growing really nicely too.

LXF: Is it the case that people in the IOT space often start out with Linux, then they run into all these constraints and Zephyr turns out to be a better fit?

KS: Well, I think Linux is definitely good in a lot of the IOT space. But there’s places where you want to have the communication down to the sensors. And that’s not really an option with Linux, it doesn’t really get any smaller than 2MB. So we can’t use it, but we want to have the same level of security and best practices around the solution at the end point. Because if your endpoint is compromised it’s [potentially putting] garbage into your ecosystem, so any analytics you do on that data, well you know what happens – garbage out.

With Zephyr we’re making sure we’ve got those secure endpoints, that we’re catching all the security updates. We have an active security team now, and have been filing CVES. In fact the 1.14.1 release came out last month – the first point release since our LTS – if you look at it you’ll find we’ve got two certifications for Bluetooth – there’s two QDIDS on it. So people using Zephyr with Bluetooth will find that a lot of their work has been done for them. We also put a CVE out with that release (CVE-2019-9506).

LXF: And I gather a big 2.0.0 release just happened, as well as the 1.14 LTS?

KS: It did, yes! With the Linux kernel you have the in-development tree where new code is added, and the important work there gets backported to the LTS release – bug fixes and security updates mostly. We’ve taken that model and applied it to Zephyr. We’re trying to learn from Linux.

LXF: So you’ve got a fancy-looking smart name badge there. Tell me about it

KS: Haha. A couple of years ago PHYTEC started getting involved in the project, and they actually built a board for it. So this is it, it’s called the Reel Board (see http:// bit.ly/lxf26reel), and they’ve made them available to some of the developers as well.

They’ve added a new module connection to the back of it so that it can run other sims – it’s running Nordic right now. You can put applications on it, and then by pressing the buttons on it you can interact and communicate with other Reel Boards by Bluetooth Mesh. It’s got a variety of sensors that are useful as well.

These kind of smart badges have applications in companies and hospitals and things like that. It’s a good prototype to start developing for.

LXF: I have a little E Ink screen for the Raspberry Pi, it’s pretty fun to do things with that, but small batteries don’t really last long with the Pi.

KS: Exactly. So with this new module, once you plug it in you can extend some of the peripherals you’re communicating with for prototyping work. You could add Ethernet or things like that. I’ve actually only just got this today, so I haven’t played with it yet. For the Embedded Linux Conference they’ve even donated a few of these boards as prizes.

The fact that we’re seeing boards like the Reel Board emerging with Zephyr on them by default, when they’re sending out the BSPS (board support packages) and SDKS means that we’re going to see a lot more Zephyr products coming out. The SDK for the Nordic Thingy 91 is also Zephyr by default, so I think we’re at the tipping point.

We knew we’d have to go there bit by bit, and that’s exactly what we’ve been doing. We’ve tried to do it in a responsible way and being community friendly as we go along. We all designed the Reel Board, and we actually got to name it. Our logo is this kite and then you’ve got a reel attached to it, so that’s quite fun.

I think there’s another version in prototype mode, so if you go down to the Stmicros booth and ask to see the Zephyr board you can see a different model.

LXF: Security and safety are key issues nowadays, and I was just (last issue) talking to my new friend Greg Kroahhartman about the Core Infrastructure Initiative (CII), in particular the badging program. I do like badges, but I digress. I gather Zephyr just earned one?

KS: Yes, everyone likes badges. One of the things that was done, probably about four years ago, was to try and figure out what pieces of software are key to our infrastructure. So they did a survey of several open source projects, and as part of that survey they came up with some criteria for what best practices are and are not. From that they developed this online badging program, where you can selfassess. So that CII program was about to be launched and Zephyr was about to be launched, so we thought “Let’s go for it”.

We got to passing pretty quickly, because we wanted to follow best practices. And then all of a sudden we stopped passing, so we were like “What happened?”. Well we’d changed our infrastructure and a lot of our links weren’t working, and the assessment process caught us. Which I consider a good sign, as the system works. Anyway we sorted that. Then later the CII introduced silver and gold level badges, so we strived for gold, basically. And now if you go on the website (https://bestpractices. coreinfrastructure.org/en/projects) you’ll see that Zephyr is one of only four gold badge-holders. So we managed to get there before Linux did.

What it’s doing is publicly documenting the process the project follows, providing evidence that shows where those processes are and making sure best practices are followed.

We’re very much focused on how we can get a good secure story emerging, from the trusted root to the sensors to the edge and to the cloud. We’re probably going to work with the Eclipse folks on some reference systems. There’s a variety of other ecosystems that are active in the IOT sphere that I’d like Zephyr to have better interactions with.

LXF: How did you get into open source? KS: I was at Motorola, and Apple had just pulled out of using Powerpc chips. We needed to sell our Powerpc boards into

the server space, so we needed to get reference Linux ports. Well first off we decided to get the GCC toolchains working and go from there. I started learning about it as a manager rather than a developer. My developer days were doing POWER optimisations back at IBM, but by that point, in the early 2000s, I was a manager. It’s probably a slightly different path than how most people get into open source.

Anyhow, we used Linux to bring these systems up, and we’d gone through the whole legal issues of getting copyright assignments to the FSF so that we could get our architecture contributed upstream. I very much believed open source was one of the saner ways of doing things and was much more supportable long term. The magic is that you do something, someone else does a little bit more, and these solutions keep getting contributed back. So there’s no need to worry about things going stale and patch porting.

From there I went to the embedded space and then to Canonical. I was Ubuntu’s release manager for two years and I learned an awful lot about open source there. When I was doing embedded stuff at Motorola/freescale there’d be about 500 packages in a BSP. So moving to Ubuntu, which has tens of thousands of packages, and trying to get all of that released on schedule was a challenge.

I learned a lot about community there too, about what works and what doesn’t work. Communities are a gift, and it’s important not to take them for granted. After that I was doing product management at Linaro, and trying to figure out where it made sense to collaborate with new areas in that ecosystem. Throughout all of that I’d actually had a project at the Linux Foundation, SPDX, which you may have heard of.

LXF: Yes! It’s all about keeping track of open source licenses in projects. I went to a talk about it a couple of years back. KS: Right, so what we’re trying to do with that is improve the software transparency of the ecosystem, so people actually understand what they’re shipping and what they have installed on their system.

I ran into issues at Freescale because we had no way of sharing license information.

We’re trying to make Zephyr a firstclass citizen in that respect. Each file is licensed, so it’s very clear which license you’re working under when you contribute to that file. Zephyr is one of the few projects, the Linux kernel is another, that’s a Developer Certificate of Origin (DCO) project. So when you contribute to it you retain your copyright, and you’re contributing under the license of the file, you don’t have to sign a Contributor License Agreement (CLA) or a copyright assignment, so it’s fairer, I think, to developers. That’s my personal opinion, but I’m very much a fan of the DCO, I think it makes a very good basis for communities. That was definitely one of the lessons we picked up from Linux.

LXF: I guess in Linux things can get really muddy, because some of those files have unclear provenance and find their way into other projects that are working under other licenses. I guess it’s nice to be starting with everything explicitly licensed in Zephyr.

KS: One of the things I’ve been working on with Thomas Gleixner, Philippe

Ombredanne and your friend Greg K-H is cleaning up the licensing in the kernel. Right now 79% of all the kernel source files have a license identifier at the top, then there’s a variety of interesting cases we’re going to be working on to finish it off.

The idea is you shouldn’t have to sit there [trying to track down licenses]. I was seeing people trying to use artificial intelligence to figure this out. Uh uh, totally attacking the problem from the wrong side. Garbage in, garbage out. Again. So fix it in the source, and let it come up from there. That way if someone copies your file, which happens all the time, that license information is there.

With Zephyr we’ve been very careful about our licensing. I do a couple of passes every now and then to make sure we’re still keeping that discipline.

MAKING ZEPHYR TRANSPARENT “We’re trying to make Zephyr a first-class citizen in that respect. Each file is licensed, so it’s very clear which license you’re working under when you contribute.”

LXF: Well I guess it’s nearly time to wrap up, and I’m not sure we can get more exciting than reindeer. But do you have other interesting applications or animals that Zephyr has been used with, or in? KS: Actually the aspect of taking Zephyr to places where it can be used within, like inside your body, is why we’re really starting to focus on safety. There’s other use cases too, but being able to get Zephyr to where you can have enough provenance associated with it and then follow the best practices in that sense, that’s a goal for us.

I’ve got a lot of friends who have medical devices implanted in them, and they want access to that source code. I think everyone has that right, whether the device is running Linux or Zephyr or something else. I think if you’ve got devices in your body you want to know what’s running on them.