Kernel Watch

Linux’s #MeToo moment means Greg Kroah-Hartman steps up while Linus steps back,

Greg Kroah-Hartman announced the release of Linux 4.19-rc6, having stepped in temporarily while Linus takes some time off for personal reflection. In his announcement, he noted that, “It all just works on my systems, and I have not heard of any major outstanding issues as of this point in time.” Indeed, by all accounts, Greg has done an outstanding job, which isn’t surprising when one considers that he’s used to maintaining the -stable kernel trees relied upon by many.

#MeToo

The Linux kernel community had an overdue appointment with reality this month. It began with a message from Linus Torvalds titled “Linux 4.19-rc4 released, an apology, and a maintainership note”, and would end with a high-profile story in TheNewYorker magazine in which several well-known developers would accuse him of years of abusive emails that had driven them out of the community.

The start was innocuous enough. An email from Linus announced the 4.19-rc4 release candidate kernel, and apologised for his “screwing up my scheduling for the maintainer summit.” He had apparently been confused between two events and almost missed the Kernel Summit until it was moved in response to his travel snafu. But there was a deeper undercurrent.

Linus said he had hoped that “maybe you can just do it [the Kernel Summit] without me.” Needless to say, the leading kernel event just isn’t the same without the leading kernel developer present, and so this notion had quickly been “overruled”. But it had also led Linus to an introspective “look yourself in the mirror” moment, in which he realised that he had been trying to avoid some issues that might come up in the discussion at such a event, chief among them his lack of empathy for others.

He said that, “This week, people in our community confronted me about my lifetime of not understanding emotions. My flippant attacks in emails have been both unprofessional and uncalled for. Especially at times when I made it personal.”

It would be easy to focus exclusively on Linus and his past behaviour in this summary. After all, he has been famously verbally abusive to many developers for a long time. A quick Google search is more than enough to reveal the kinds of invective that has been levelled over time (much of which is not printable).

That could be used to create a narrative, certainly. But focusing solely on the behaviour of Linus Torvalds, and the (apparently genuine) desire he may have to change his ways for the better would not do enough justice to the hard work of those who are seeking to make a positive and lasting change. Developers such as Sage Sharp and Valerie Aurora (who feature in

TheNewYorker piece) have tried for years to foster a more collegial atmosphere, but their efforts have all too often been rebuffed. As a result, both they and others have chosen to leave the community rather than stick around and continue to put up with what they saw as behaviour that never changed.

The bottom line is that where we ended up wasn’t really good for anyone. After all, the developers who have left have been behind some really cool technologies that we all use and rely upon every day. That USB device you’re plugging into your laptop? That container image you’re using with an overlay filesystem? These are just two examples of technologies developed by people who have since left the community because they didn’t feel welcome.

And they’re not alone. It’s difficult as one who usually covers technology to address this topic without appearing to become political or to take sides, but it’s important to share that anecdotally there are a number of people who would choose to be involved with Linux kernel development today who do not do so because they just don’t want to have to deal with the kind of behaviour that’s occurred on occasion over their years of involvement.

At the same time, there are many wonderful people involved in the kernel, many of whom have hearts of gold and are among the nicest humans alive. As I said previously, it would be easy to focus exclusively on Linus, but the truth is that there’s a positive wave of change happening throughout the world we live in as more people find their voice and speak out, and it’s indeed a positive sign that Linus embraced the opportunity to change.

His mail included a new Code of Conduct, which replaces the older (and much maligned) Code of Conflict. In the new document, a pledge is made by the maintainers to foster “an open

and welcoming environment” by adhering to various standards and responsibilities that are enumerated. Enforcement for the new CoC is to be handled by the Technical Advisory Board (TAB), an elected group of representatives from across the kernel community.

Linus will return to kernel development in a few weeks, for the 4.20 cycle. However, only time will tell whether his reflections will translate into lasting changes.

Ongoing development

Kees Cook (@ kees_cook) tweeted “The last of the crypto VLAs removed! Lots of people helped get rid of kernel VLAs (at least 108 commits by 15 people across four kernel versions)”. This is good news for those relying on the robustness of Linux cryptography.

VLAs (Variable Length Arrays) are what they sound like: a (mis)feature of the C language in which the programmer need not pre-determine the size of an array containing data objects, but instead rely upon the compiler to automatically allocate storage on the stack. However, kernel stacks are extremely limited in size, and overrunning them tends to result in Very Bad Things Happening Very Quickly. Thus VLAs were always dangerous in kernel code, but especially in sensitive-security code.

Meanwhile, the Amazon team posted a patch titled Coscheduling for Linux, which seeks to extend the Linux scheduler, known as CFS (completely fair scheduler) with support for limiting which processes (known as tasks within the kernel) can be coscheduled onto the same underlying core at the same time.

Many modern processors, including x86 cores, include support for a technique known as SMT (Simultaneous MultiThreading) invented by Susan Eggers (incidentally the first woman to receive the prestigious EckertMauchly Award for Computer Architecture, just this year) back in the mid-90s. This enables two threads to run at the same time on logical processors that share certain CPU resources, such as the L1 data cache. Threads that share a core can cooperate, but they can also compete, or even interfere with one another, and so there are reasons why it can be advantageous to prevent coscheduling them together.

This is made doubly true in the case of the “L1TF” (L1 Terminal Fault) CPU vulnerability. On impacted CPUs, it’s important not to run two untrusted virtual machines on SMT threads (called Hyperthreads in Intel’s implementation) of the same core at the same time. The Coscheduling patch is meant to prevent this. The chances of these patches being merged as-is are almost zero, but they should hopefully revive a discussion about the best upstream solution.