Calamares installer...............
Perhaps the greatest development since automatic partitioning, Mayank Sharma lifts the lid on Calamares to figure out what’s got the distros excited.
Discover the inner workings of one of the most accomplished distro-agnostic installers, with Mayank Sharma as your happy guide.
The diversity of Linux installers is a nuisance. Although navigating a Linux install is a smooth experience these days, the process still presents many avenues for new users to get things wrong. The lack of a unified user experience for installing a distro only gives them another opportunity to bungle up.
It isn’t just the users who suffer. Smaller distros spend considerable time adapting the upstream installer as per their requirements. “Many distributions rely on volunteer contributions, so in their quest to provide a great user experience in all areas of personal computing, they often don’t have the resources to develop an installer. This is unfortunate, because the installer is a user’s first encounter with a distro, so it can make or break the whole experience,” believes Teo Mrnjavac. The situation bothered him enough to start hacking a distribution independent installer framework in June 2014 called Calamares.
Before Calamares came about, using a customised and rebranded version of a mainstream distro installer was the norm. Teo explains: “Usually every distro would have a different set of customisations and patches, painstakingly collected and repackaged, and sometimes based on an old version of the mainline product.” This situation often led to a shoddy installation experience. Teo observed that the majority of what a distro installer does is the same for all distros, while some of it is different and distro-specific. He wondered: “Could there be a way to take care of the common stuff for all distros, and also provide them the tools to sort out the distribution specific bits themselves?”
The Calamares project began with the goal to provide distros with a single distribution-agnostic installer that can be easily branded and customised as per their requirements. In the three years since Teo began working on Calamares, the installer has been adopted by dozens of distros.
“A user’s first encounter with a distro’s installer can make or break the whole experience”
Before Teo could start coding the installer, there was one key hurdle he had to overcome. “Perhaps the most important challenge early on was to get the design bits just right,” he says.
“Mistakes in that early phase could have resulted in problems months and years down the line. I’m very satisfied that the idea to make everything a plugin and support for both C++ and Python proved to be a huge success. This allowed us to integrate bits and pieces of Python code from other installers, and to keep a low barrier to entry for contributions.”
Adriaan de Groot, the current maintainer of Calamares, says: “I’m happy to make use of the KDE Visual Design Group (VDG) to bounce ideas off, since my own design skills are extremely limited,” he says. “When visual bugs are reported, I ask people to draw big red arrows in screenshots, because I just don’t see some bugs that others find glaring mistakes.”
Design isn’t the only non-technical issue that the project grapples with. “Calamares is an installer product, but it’s also a community,” says Teo. “When a distribution maintainer comes into the IRC channel asking for help on customising Calamares for his or her distribution, it’s important to provide a friendly environment and high-quality support.”
Adriaan adds that the project also needs to continuously evaluate its target audience: “In a sense, Calamares has distros as the target audience and they get to decide how to configure Calamares for their own target audience,” he says. “To do that, Calamares needs to support all the different target audiences that downstreams have. So sometimes we end up with a discussion on whether something is relevant – or even if it’s something we should support at all.”
Adriaan explains his position regarding Calamares helping users with advanced cryptography setups: “While I understand that there are people who want (and need) that, I think that audiences who need that kind of setup, should also be competent enough to set it up without Calamares’ help.” Yet he’s quick to add that this position might change as their target audiences and feasible features list evolve.
In terms of technical challenges, Teo points to partitioning as a significant one: “Over the years, the partitioning module was where I spent most of my time. It’s not easy to make something like partitioning reliable, flexible and user-friendly.”
Adriaan concurs that partitioning is a big challenge. However, he adds that the project has a good working relationship with the maintainer of KPMcore, which is the KDE library for managing partitions that also provides a usable APIm used by the Calamares project: “Code moves back and forth between Calamares and KPMcore. Sometimes one moves ahead faster than the other. Right now we’re moving more stuff into KPMcore.”
(K)alamares?
Calamares’ ties with KDE go far beyond the use of the KPMCore library. A majority of Calamares developers, including Teo, are KDE developers and the project is
sponsored by Blue Systems, a German IT company that also supports the development of several popular KDE projects. In fact, Kevin Kofler, the lead developer of Kannolo, which is a pure KDE Fedora remix, admits he gravitated towards Calamares because it uses Qt and doesn’t depend on any GTK+ libraries.
Another distro that makes use of Calamares’ Qt lineage is Nitrux. According to its developer, Alexis Lopez Zubieta, although the distro is based on Ubuntu, he wasn’t able to use Ubiquity “to distribute Nitrux properly”. Back in 2015, Alexis settled on Systemback before a two-year hiatus in the distro’s development cycle. By the time Nitrux was reanimated, Systemback was defunct and Calamares was being used by many other KDE distros: “Calamares’ ease of use trumps over Systemback and although both are developed in Qt, Calamares also uses QML and that’s an important plus for us.”
“Calamares uses a few select KDE frameworks, mostly in the partitioning module. This is because the partitioning module shares a backend with the KDE Partition Manager,” says Teo. When asked what this means to non-KDE distros, Teo adds that distros that don’t ship other software by KDE on their live system will need to ship a few KDE Frameworks libraries if they plan to use Calamares.
While it might sound like an unnecessary burden, Adriaan puts it into perspective. The main Calamares installer depends on three KDE Frameworks libraries that are just over 900k in total. The Calamares modules have their own additional dependencies, with the biggest one being KDE Partition Manager. This has recently dropped two massive KDE Frameworks libraries after concerns raised by Calamares about those dependencies, reveals Adriaan. Kevin reaffirms: “The dependency footprint of Calamares is actually smaller than the one of Anaconda, so I didn’t have to drop any packages to make room for Calamares over Anaconda.”
Furthermore, Teo says that “the GUI components of Calamares don’t rely on a KDE Plasma environment in any way, and we’ve always taken great care to make sure Calamares looks good and works well on all desktop environments.” Adriaan agrees: “Calamares is also used in nonKDE environments, and we do pay attention to these, to keep Calamares as a universal installer.”
This is perhaps why Calamares is used by other non-KDE distros as well. One such distro is SwagArch, which ships with the Xfce desktop. Its lead developer Mike Krüger tells us that he has no issues with the Qt/KDE dependencies of the Calamares installer. After the distro is installed, the Calamares installer and its dependencies are zapped completely.
Building blocks
The Calamares installer describes itself as a framework, rather than a finished product, that enables distributions to compile the installer itself. Think of it as a modular tool that includes lots of pluggable modules for various system installation activities such as partitioning, setting up users, encrypting filesystems and more. Calamares has two kinds of modules. The ‘view’ modules help draw the pages in the installer that are visible to the user, while the ‘job’ modules do the grunt work behind the scenes.
The individual distros can mix and match any of the Calamares modules that are written in either C++ or Python. “The module system makes it easy to just enable the modules you need and disable the ones that don’t apply to your distribution, and distribution specifics within a module can be easily set through configuration files,” says Kevin. He’s especially grateful for the netinstall module, which enables distribution maintainers to ship minimal ISOs with only a basic set of preinstall packages. The netinstall module gives users the option to install groups of packages from a
“It’s easy to enable modules you need and disable the ones that don’t apply”
predefined list during installation. By using the module, the distributions can choose which groups to display and which packages should be in those groups.
Kevin also prefers the Calamares partitioning module that gives the user more control than the default Anaconda partitioning spoke. “However, recent releases of Anaconda now integrate the Blivet GUI as an alternative to the default partitioning interface, so this isn’t an issue anymore,” he says. “Now, the remaining user interface differences are mostly down to subjective design decisions. But the [KDE/Qt] dependencies and the netinstall module still remain as strong reasons to use Calamares in Kannolo.”
In fact, the Calamares community has mushroomed around the module infrastructure. “New modules are still developed, and downstream continues to provide fixes, extensions and features for existing modules. Certainly no one uses them all at once, but all of them are used by somebody,” says Adriaan.
Boot issues
Kevin brings up one notable issue, namely the grub2-efi-modules package which runs the grub2-install command on UEFI systems. This additional package is needed because Calamares doesn’t support Shim (a first stage bootloader that embeds a self-signed CA certificate) that Anaconda uses to set up GRUB2 on UEFI systems.
It’s issue like these that make up most of the discussions on the installer’s IRC channel (#calamares on Freenode). “There are a half-dozen downstream people on the Calamares IRC channel all the time, providing feedback, providing extra eyeballs on the code (because I do make mistakes, and am grateful when those get caught), chatting about packaging and possible features,“says Adriaan.
On the subject of user feedback, Adriaan shares that the total reported issues, including both bug reports and feature requests, are evenly split between individual users and downstream distros: “I try to talk things over with multiple distros before settling in to an approach. The “support them all” approach, which is inclusive of all the distro’s, sometimes leads to messy code as we handle all the different variations downstreams have come up with, but it’s worth it,” he says. “I’m ever so happy to see Calamares as a small part of any new Linux distro.”
Onwards and upwards
Teo is no longer involved with Calamares but he’s proud of the community that has fostered around the project. Talking about his five-month old tenure as the new maintainer, Adriaan is still trying to get a grip on all the bits and to make sure the community remains happy. “There’s some purely personal long-term things I’d like to achieve (which involve Calamares and FreeBSD, so moving the “universal” beyond Linux),” he reveals. “Whatever vision evolves over the next six months or a year will probably be an extension of, and an evolution on, Teo’s original concept.”
In its brief existence, Calamares has managed to achieve its developer’s vision. In addition to smaller distributions, even corporate sponsored projects like Fedora offer Calamares in their repositories for downstream distros. While the project continues to evolve and take on new features it has helped bring order to the chaos of installing Linux, especially for the countless scratch-their-itch distros.
While talking about the benefits Calamares brings to Nitrux, Alexis sums up what the installer has managed to achieve for the larger distro community: “Calamares offers a solid foundation upon which we can build a user-friendly way to distribute Nitrux. Their QML and Qt framework enables us to integrate the installer seamlessly into the desktop so the user is right at home when clicking through the installer steps,” he says. “This integration means that it doesn’t look ‘alien’ to the rest of our custom themes. Its friendliness also means that we can customise it just as we want or need.”