Config management SaltStack...........................
Chris Notley will show you how a configuration management tool called Saltstack, which was made for enterprise, can be put to good use at home.
Chris Notley explains how this configuration management tool can be used at home.
The days of a single shared family PC in the living room are long gone – whether a child or partner’s laptop, a NAS or even a Linux-based media player such as Kodi – most readers will have more than one computer at home. In addition to PCs at home, the letters pages of LXF also show that many readers have helped convert friends or family over to Linux, which often means providing remote support. This tutorial introduces the concept of configuration management and how it can help you at home.
Many configuration management tools were created by sysadmins in large environments, often where hundreds (or even thousands) of servers need to be managed. Manually applying even a minor configuration change across hundreds or thousands of servers would be a huge task. The main purpose of all configuration management tools is to streamline repeated actions across multiple systems and ensure consistency. If you’re regularly making fresh installs of Linux onto PCs or laptops at home, ensuring that your preferred packages always get installed consistently could save wasted time and frustration. For this tutorial, we’ll be using a tool called Saltstack (or ‘ Salt’ as it is more commonly known), which is open source under the Apache 2.0 licence. The default configuration of Salt uses the concept of Master and Minion roles: the former is responsible for storing configuration settings and pushing them out to managed systems, while the latter is delivered by a small agent running on each system you want to manage. The Master role needs Linux or Unix, whereas the minion role can be easily installed on virtually any flavour of Linux/Unix and even Windows.
If you already have a PC configured as a server and running Linux, this would make an excellent choice for your Salt Master. The remainder of our tutorial will make reference to a lab consisting of three Ubuntu PCs, one acting as both Master and Minion and two acting just as Minions ( as shown in the diagram, p81):
Installing Salt
The latest LTS version of Ubuntu (16.04 or Xenian Xerus) has a recent version of Salt available in the default software repositories (repos), but for older versions of Ubuntu the Salt community maintains a Personal Package Archive (PPA), making it very easy to install. The code below shows the process of installing the Master and Minion roles onto an Ubuntu 14.04 PC: $ sudo add-apt-repository ppa:saltstack/salt $ sudo apt-get update $ sudo apt-get install salt-master salt-minion
If you don’t have the apt-add-repository command on your system, run the command $ sudo apt-get install pythonsoftware-properties .
The process on our second PC – ‘minion1’ and also running Ubuntu 14.04 – is virtually identical to that shown above—we just omit the salt-master package (ie $ sudo aptget install salt-minion ). On our third PC ('minion2') the process is even easier as it is running Ubuntu 16.04 and hence doesn’t need the PPA added, as shown below: $ sudo apt-get update $ sudo apt-get install salt-minion
You’re now ready to configure your Minions and tell them which Master server to talk to. By default, the Minions will attempt to resolve the hostname ‘salt’ to an IP address and associate with it. It’s easy to change this behaviour using the Minion configuration file (/ etc/salt/minion on most systems). Traffic between the Master and Minions is encrypted using keys generated when the software runs for the first time. The default behaviour of a Minion is to accept any Master encryption key, but it’s generally best practice to explicitly define the key fingerprint, which you can check by
running the command sudo salt-key -F on the Master server, which should show something similar to the following: Local Keys: master.pem: c6:34:4c:4d:ad:e7:28:80:92:78:99:96:6d:e4:a8:b5 master.pub: 52:b1:64:3c:84:61:82:b0:2b:ca:78:e9:e1:62:e3:af Take a copy of the fingerprint of master.pub (starting 52:b1… in the example above) as you will need this for your Minion configuration. You are now ready to configure the Minion. First, open up the configuration file using the text editor you are most comfortable with (eg sudo vim /etc/salt/ minion or sudo nano /etc/salt/minion etc) and add the following lines: master: 172.16.65.10 master_finger: ‘52:b1:64:3c:84:61:82:b0:2b:ca:78:e9:e1:62:e3: af’
The first line can contain either an IP address or hostname and should be adjusted to match your own Master server. The second line contains the encryption key fingerprint copied earlier from our Master. With all three Minions configured, you can now restart the software (depending on your Ubuntu version, using one of the following sudo service
salt-minion restart , sudo /etc/init.d/salt-minion restart , or sudo systemctl restart salt-minion ). Once restarted, each Minion will communicate with the Master, which will initially place the Minions in the unaccepted state ( seetheSalt Security for more details, p83). This can be confirmed by running the sudo salt-key -F command on your Master, as shown below: Local Keys: master.pem: c6:34:4c:4d:ad:e7:28:80:92:78:99:96:6d:e4:a8:b5 master.pub: 52:b1:64:3c:84:61:82:b0:2b:ca:78:e9:e1:62:e3:af Unaccepted Keys: saltmaster: 56:85:90:8e:78:34:e3:f4:06:ff:60:c0:d2:bd:69:8b minion1: 61:bb:48:82:25:38:83:bc:ad:da:1b:da:34:1e:b4:8a minion2: 5d:5c:bf:d8:83:27:91:e8:7e:91:a1:95:fb:e8:7d:48 In our lab, we can accept all the keys in one go using the sudo salt-key -A command, as shown in the example below: The following keys are going to be accepted: Unaccepted Keys: minion1 minion2 saltmaster Proceed? [n/Y] y Key for minion minion1 accepted. Key for minion minion2 accepted. Key for minion saltmaster accepted.
Sending commands
You should now have a functioning Salt installation with just a few lines of configuration code. You can test connectivity between the Master and Minions by issuing a command. Commands can be targeted at individual Minions or groups (either by wildcard or shared properties, such as the operating system and CPU architecture etc.). You can initially test communication using the command sudo salt “*” test. ping command, which verifies each defined Minion. The output of this command using the earlier defined test lab is shown below: minion1:
True saltmaster:
True minion2: True Salt comes complete with a large number of built-in modules and we will try the disk module now to quickly check the amount of space used on each of our Minions’ root volume. Issue the command sudo salt “*” disk.percent / on your Master; the results for our test lab are shown below: minion1:
14% saltmaster:
14% minion2:
27%
Clearly, the above example is not very exciting and many readers might suggest it’s just as easy to SSH to a remote system and run the df command. However, imagine that remote system is your Dad’s Linux PC sitting behind a broadband connection where the IP address changes regularly. In such a case, you might have to first talk your Dad through the process of checking his broadband IP address before you could connect to his PC, which can be a painful process. It’s also possible to issue remote shell commands on your Minions using the cmd.run (eg sudo salt “*” cmd.run
“apt-get clean” to clear space on all Minions).
Scripting It
While Salt can provide a useful way of running interactive commands on remote hosts, it really comes into its own when you take advantage of its state management capabilities. State management is all about defining a standard set of configurations that can be applied to either individual PCs or groups of PCs. The state management of Salt is very powerful but initially a little intimidating for users without experience of such tools, but we’ll guide you through the process. All of Salt’s state management is controlled by text configuration files on the Salt Master server. The files contain directives made up of the modules already mentioned earlier (eg installing packages, updating configuration files and creating users etc). These can then be applied to all PCs, individual PCs or groups of PCs.
Rather than continue to talk hypothetically about this functionality, we will crack on and start using it to show how easy it is to apply settings automatically. In order to do this, we will need to add some settings to our Salt Master server
and enable its file-serving functionality. Open up the Master configuration file using the text editor you are most comfortable with (eg sudo vim /etc/salt/master , sudo nano /
etc/salt/master etc) and add the following lines to the bottom of the file: file_roots: base: - /srv/salt
Salt uses the YAML markup language for all of its configuration files, which requires careful formatting and syntax. Where commands are nested as shown above, you must use exactly two spaces (for each nested level) and never use tabs. If you ever receive an error when trying to make changes, always check first to make sure that your spacing is correct. With your Master configuration updated, you now need to create the base directory using the command sudo mkdir -p /srv/salt and once you have done this, you can restart the salt Master ( sudo service salt-master restart , sudo /etc/init.d/salt-master restart or sudo systemctl restart salt-master depending on which version of Linux you are running).
For our first example, we will ensure that a few nonstandard packages are installed on every system. The packages in question are nmap, iperf and tshark and are not installed by default on Ubuntu (server or desktop flavours). They are tools I use infrequently, but invariably remember they are missing when away from internet access and need them.
We will install these packages by creating our first Salt formula, which we will call netutils. You need to create a directory to contain the formula using the command sudo mkdir /srv/salt/netutils . Then using the editor of your choice, create a file called init.sls inside this new directory (eg sudo vim /srv/salt/netutils/init.sls , sudo nano /srv/salt/netutils/ init.sls etc) and enter the following lines: pkgnetutils: pkg.latest: - pkgs: - iperf - nmap - tshark The first line of the file is an arbitrary name or label for the task—the name must be unique though, so making it descriptive is sensible. The second line (2-space prefix) is the module we want to run; in this case stating we want packages to be installed. The third line (4-space prefix) tells Salt that a list of packages will follow and the remaining lines list each of the packages that need to be installed (each with six spaces at the beginning and a dash (-) to indicate they are list items).
We are now ready to create our top file, which is what tells Salt which formulas should be applied to which Minions. Using the editor of your choice create a file called top.sls inside the base Salt directory (eg sudo vim /srv/salt/top.sls and sudo nano /srv/salt/top.sls etc) and enter the following content: base: ‘*': - netutils The first line defines the environment, which in our case is the default catch-all base . The second line (2-space prefix) targets the Minions we want to install, which in this case applies to all Minions ‘*’ . The third line (4-space prefix)
defines the formula we want to apply to the target, which in this case is the ‘netutils’ formula we have just created.
You can now apply the configuration with one simple command: sudo salt “*” state.highstate . This command instructs all Minions to contact the Master server and pull down the top.sls file, identify any tasks that relate to them and apply those that do. The output of the command will be quite verbose (it will list each package installed on each Minion in turn including all dependent packages). An example of this output is shown in the screenshot right: While it’s useful to be able to apply configurations to every PC you manage, you may also want to target specific groups of hosts. Salt offers numerous way to target Minions, the most interesting of which are ‘grains’. Minions expose certain grains automatically (such as the Linux distro, version and CPU architecture etc), but you can also define your own custom values. We will use this now to define a role of ‘media’ to two of our Minions and install both Audacity and Handbrake to that role.
On both minion1 and minion2, open the Minion configuration files with the editor of your choice (eg sudo vim /etc/salt/minion , sudo nano /etc/salt/minion etc) and add the following lines to the end: grains: roles: - media After restarting the Minion software ( sudo service saltminion restart , sudo /etc/init.d/salt-minion restart or sudo systemctl restart salt-minion depending on your Ubuntu version) you will need to create a new formula on your Master called ‘media’. Like for our earlier ‘netutils’ example, create a directory for the formula using the command sudo mkdir / srv/salt/media and then using the editor of your choice, create a file in that directory called init.sls (eg sudo vim /srv/ salt/media/init.sls , sudo nano /srv/salt/media/init.sls etc) and enter the following lines: pkgmedia: pkg.latest: - pkgs: - audacity - handbrake With your new formula created, the last step is to edit the Salt top file with the editor of your choice (eg sudo vim /srv/ salt/top.sls , sudo nano /srv/salt/top.sls , etc) and append the following lines to the bottom: ‘roles:media':
- match: grain - media The first line (2-space prefix) indicates that this matches Minions with the role ‘media’. The second line (4-space prefix) specifies that the match should be targeted using grains and the last line (4 space prefix) specifies which formula should be applied (in this case, ‘media’). You can now apply the changes by running the sudo salt “*” state.highstate command on the Master. This time around you will notice that the ‘netutils’ formula was already applied, but on minion1 and minion2 only, the new ‘media’ formula was applied, and both Audacity and Handbrake were successfully installed ( as picturedabove).
To conclude our tutorial, we have built a working installation of Saltstack with little more than 20 lines of configuration on our Master and a few lines on each Minion. By using a tool, such as Salt, it’s possible to easily and quickly add any additional software you might want on your PCs. Rather than having to remember all the extra software every time you install a new version of Linux or add a new PC, you only need to install the Minion software and with one command from the Master, have everything ready to go!