Amazon adopts KVM: what’s next?

Bold statements on news sites and Assembler at re:Invent 2017 slides. What’s going on, really?

Regular readers of my Administeria instalments will have most likely heard of virtualisation. And that Amazon moved away from Xen, too. Companies change their attitude to software from time to time, even the big ones – remember the “Microsoft loves Linux” news from a few years back? So, what’s the fuss?

To understand it better, let’s revisit the history of computer virtualisation in x86. Virtualisation itself is nothing new – it’s been around since the 1960s. But for an architecture to be virtualisable, all of its control-sensitive instructions (such as disabling interrupts) must also be privileged. x86 wasn’t fulfilling this requirement for a long time, and pioneers such as QEMU or VMware resorted to emulation (which was slow) or were employing clever tricks such as binary translation (which was complex and also slow).

Xen debuted in 2003 with a simple idea. If we can’t make a hardware to trap control-sensitive instructions, let’s make a guest kernel and not use them at all. Instead, the kernel would issue hypercalls much the same way unprivileged processes carry out system calls in Linux. This is already faster (overheads are as low as 10 per cent), but it doesn’t work if you can’t make the OS co-operate (think early 2000s Windows). The latter wasn’t much of a concern to early Amazon Web Services: Xen was a mature technology by the time of launch, so it adopted it.

Finally, x86 introduced hardware virtualisation support (Intel VT-x and AMD SVM) around 2006. Xen added support for these technologies quickly, but other projects arose, most notably, KVM. Where Xen was doing everything by itself as a separate project, KVM melded with the Linux kernel and benefited from its improvements.

Soon afterwards, Red Hat and others switched to KVM. Amazon (and Citrix) were perhaps two major players shipping Xen in 2017. That’s why a “lightweight” KVM-based Nitro Hypervisor made such big news. Note, however, that Nitro doesn’t reuse other KVM ecosystem bits like QEMU or libvirt. In addition, Xen will keep powering older AWS instance types for their lifetime. But Amazon made it clear that Nitro is the future, and Xen is now more of a legacy than a mainstream.