Ten Effective Linux Commands for Systems Administrators

Systems administrators need a bag of tricks to ensure that everything runs smoothly without any hitches. Linux has a fine set of utilities and commands to assist sysadmins in their task. Mastering these tools will take the efficiency levels of Linux admin

GNU/Linux is one of the most popular operating systems for servers. Today, most of the operating systems use advanced and modern graphical user interfaces (GUIs) but the CLI (command line interface) is still popular. Using the CLI/scripts, you can automate complicated tasks and execute them in a repetitive manner. In this article, we will discuss the most common CLI utilities. If you are familiar with GNU/Linux and want to become more productive, then this article is for you.

1) find

Searching for a file in a file system is a very common task and we have to do it quite often every day. GNU/Linux provides the find command which searches for files in a directory hierarchy. Given below is the syntax of the find command:

find [STARTING-POINT] [EXPRESSION]

In the above example:

STARTING-POINT represents the directory’s location, from where the search will start. Note that if STARTINGPOINT is omitted, then the search will begin from the current directory.

EXPRESSION is evaluated to search the file. EXPRESSION can be a name, type, size, permission, owner, and so on.

Let us search for a file with a given name. Here, we’ll be using the file name as an EXPRESSION.

$ find src-dir -name hello.txt

In the above example, src-dir is the STARTING-POINT and hello.txt is the EXPRESSION. Here the -name option indicates that we are searching for the file by name. If you want to perform a case-insensitive search operation, then use the -iname option instead.

For the find command, EXPRESSION can be a pattern as well. Many times we want to search certain types of files like .txt, .jpg, .mp3 and so on. The example below shows how to use a pattern in EXPRESSION:

$ find src-dir -name “*.txt”

The above example will search and list all .txt files recursively from src-dir.

We can use the file type as an EXPRESSION with the find command. For instance, use the command below to search only directories:

$ find src-dir -type d

In the above example:

The type option indicates we are performing a search

based on the file type

Argument d indicates the directory file type

In addition to the directory, the find command supports the following file types:

b: block device c: character device f: regular file l: symbolic links p: named pipe s: socket

The find command allows you to perform a search based on the file size. We can provide an EXPRESSION which will compare files that are greater than, less than or equal to the provided size. We can also perform a search based on permissions, for which we have to use the -perm option. For instance, the command below searches for files with the 664 permission:

$ find src-dir -type f -perm 664

The find command allows you to perform some additional operations while searching. For instance, it provides the -delete option, which will remove a file that matches with EXPRESSION. The following example shows the usage of the -delete option:

$ find src-dir -type f -size +2k -delete

In the above example, all files greater than 2KB in size will be deleted.

We can also execute bash commands while performing a search. We can achieve this using the -exec option.

2) diff

We often need to compare the contents of files. Doing this manually is a tiresome and error-prone task. But fortunately, GNU/Linux provides a command for this, which will compare files, line by line, and report if any differences are found. When the diff command is combined with the patch command, it makes a powerful combination. With this command, we can apply changes from one file to the other. This section describes both these commands.

First, create two files with the following content:

# file-1: version1.txt str1 str2 str3

# file-2: version2.txt str1 str3 Now compare these files using the -u option, which stands for unified diff:

$ diff -u version1.txt version2.txt

--- version1.txt 2017-12-30 14:06:38.120849370 +0530 +++ version2.txt 2017-12-30 14:06:46.976750148 +0530 @@ -1,3 +1,2 @@ str1

-str2 str3

The above output shows that the line ‘str2’ is not present in the version2.txt file. We can store this diff output in a file and apply it as a patch. To create a patch file, just redirect the output to some file as shown below:

$ diff -u version1.txt version2.txt > diff.patch

If we apply this patch to the version1.txt file, then it will remove the ‘str2’ line from this file. The example below shows this:

$ patch -p1 version1.txt < diff.patch patching file version1.txt

$ diff -u version1.txt version2.txt

After applying the patch, both files will be identical; hence, the diff command does not show any differences here.

To revert the patch, execute the commands given below and follow the on-screen instructions shown below:

$ patch -p1 version1.txt < diff.patch patching file version1.txt

Reversed (or previously applied) patch detected! Assume -R? [n] y

The combination of diff and patch is really powerful. Many version control systems like Git, Subversion and CVS use this feature.

3) rename

Renaming multiple files is one of the common tasks of a sysadmin. GNU/Linux provides the rename command which will serve our purpose. It is particularly useful when we want to rename multiple files with a specific pattern. For instance, the command below renames all .TXT files to .txt:

$ rename ‘s|.TXT|.txt|’ *

Storing similar types of files in a directory is also a very common task. We can do it very easily with a combination of find and the mv command. The command below moves all MP3 files to a target-dir directory:

$ find src-dir -type f -name “*.mp3” -exec mv {} target-dir \;

4) tar

Sometimes it is convenient to operate on a single file rather than multiple files and here, the tar command comes into the picture. tar is a short form for ‘tape archive’. As the name suggests, it is an archiving utility that stores multiple files into a single one. Given below is the syntax of the tar command:

tar [OPTIONS] [TAR NAME] [FILES TO BE INCLUDED IN TAR]

To create a tar bundle, execute the command given below in a terminal:

$ tar cvf archive.tar 1.txt 2.txt 3.txt

In the above example: c option stands for create archive v option stands for verbose mode f option stands for file names mentioned for archive

The tar command allows us to manipulate tar bundles without recreating them again. For instance, to add a new file into an archive, use the -r option as shown below:

$ tar rvf archive.tar 4.txt 4.txt

$ tar tf archive.tar #list the content of tar file 1.txt

2.txt

3.txt

4.txt

By default, tar only archives multiple files; it doesn’t do any compression. There are various compression utilities available like bzip2, gzip, zip and so on. To compress a tar bundle using bzip2, execute the command show below:

$ bzip2 archive.tar

After compression, it will append the .bz2 extension to the tar bundle. If you compare sizes, before compression, the tar bundle was 20KB and after compression, it gets reduced to 4KB, as shown below:

# Size before compression $ ls -sh archive.tar

20K archive.tar

# Size after compression $ ls -sh archive.tar.bz2 4.0K archive.tar.bz2 5) fdisk

We partition disks for better management and utilisation of available storage. GNU/Linux provides the gnome-disk utility which is a GUI based application. However, we can do similar things with fdisk, which is a CLI based utility and can be used to manipulate the disk partition table.

Manipulating disk partitions recklessly will cause data loss; hence, we are going to use the fdisk command with a pseudo disk. We’ll use a file as a disk, using the losetup command. Perform the steps given below to create a pseudo disk.

First, create a file of size 200MB using the dd command:

$ dd if=/dev/zero of=disk.img bs=1M count=200

Next, set up this file as a loop-back device so that, hereafter, we can use /dev/loop0 as a device:

$ sudo losetup /dev/loop0 disk.img

We can perform various actions with fdisk like printing the partition table, creating new partitions, deleting existing partitions, writing the partition table to disk, and so on. Let us perform all these actions, one by one.

To start the fdisk utility, use the command given below:

$ sudo fdisk /dev/loop0

After entering the above command, you will be shown a welcome message and the system will wait for a command to be entered. The section below describes various actions that can be performed using fdisk.

Print the partition table

To print the partition table, type p and press Enter. This will display information about the disk and its partitions. As we haven’t created any partition yet, it will show only information about the disk.

Create a new partition

To create a new partition, type n and press Enter. Then follow the on-screen instructions.

Validate a created partition

To view the created partitions, type p and press Enter. Delete a partition

To delete a partition, type d and press Enter. Follow the on-screen instructions to complete the procedure.

Write a partition table

To make the changes permanent, we need to write this partition table to the disk. Type w and press Enter to complete this action.

Quit

Type q and press Enter any time to quit the fdisk utility.

6) Networking related commands

Networking is an essential part of a computer system. However, it is complex and can be unstable sometimes. This

section discusses a few open source utilities that will help to debug networking related issues.

ping

We can use the ping command to check connectivity between hosts. It uses the Internet Control Message Protocol (ICMP) to check connectivity. Given below is the syntax of the command:

$ ping [ADDRESS OF HOST]

If you are connected to the Internet and the host is reachable, it’ll start displaying ping statistics. Press Ctrl+c to abort it. We can also specify the packet count for ping. It’ll stop automatically after sending count packets. For instance, the command below will stop after sending four packets:

$ ping -c 4 google.com

host

This is the DNS lookup utility, which can be used to convert host names to IP addresses and vice versa. For instance, the command below prints all the IP addresses attached to the google.com domain: $ host google.com google.com has address 216.58.203.206 google.com has IPv6 address 2404:6800:4009:806::200e google.com mail is handled by 30 alt2.aspmx.l.google.com. google.com mail is handled by 10 aspmx.l.google.com. google.com mail is handled by 50 alt4.aspmx.l.google.com. google.com mail is handled by 20 alt1.aspmx.l.google.com. \google.com mail is handled by 40 alt3.aspmx.l.google.com.

Alternatively, you can also use the nslookup utility for DNS lookup.

route

The route command is used to display routing table information. This table is maintained by the operating system. Execute the command given below to display the routing table on your host:

$ route -n

traceroute

When we send a packet from source to destination, it may travel through multiple gateways. If we want to find those intermediate gateways, then we can use the traceroute command as follows:

$ traceroute -n google.com The above command will show all the intermediate gateways between your host and google.com.

7) wget

Often, we download contents from the Internet/network. Most of the time, we use the browser to do this. However, GNU/Linux provides the wget utility, which can be used as a network downloader. This section describes a few examples. Given below is the syntax of the wget command:

wget [OPTIONS] [URL]

While downloading, it displays the progress bar, which shows the following:

-- Percentage of the download completed

-- Total amount of bytes downloaded so far

-- Current download speed

-- Remaining time to download

Like other utilities, wget is also a powerful utility. It provides various facilities to make our life easy. If your Internet connection is not stable, then downloading may be interrupted. In that case, we can provide a retry count. For instance, in the example that follows, we have provided the retry count as 3:

$ wget -t 3 <URL>

It’ll retry three times before throwing out an error. To provide infinite retries, set the retry count to 0 as shown in the example below:

$ wget -t 0 <URL>

Being a very flexible utility, wget can also be used to restrict the downloading speed. It provides the --limit-rate option for this. For instance, use the command given below to set the downloading rate to 512KB:

$ wget --limit-rate=512K <URL>

One of the nice things about wget is that if downloading is interrupted, then it can be resumed from that point. Use the command given below to resume an interrupted download:

$ wget -c <URL>

8) Working with a remote host

We often interact with remote hosts to download or upload content. This section discusses command line utilities that will perform these tasks.

scp

One of the common tasks is to transfer files between the remote and the local host. GNU/Linux provides a remote

copy program, namely ‘scp’, which stands for ‘secure copy’. It uses ssh for data transfer, and uses the same authentication and provides the same security as ssh. Given below is the syntax of the scp command:

scp [OPTIONS] user@src-host:/src-dir user@dst-host:/targetdir

To copy the contents from a remote host to the local one, execute the command given below in a terminal:

$ scp -r user@remote-host.com:/remote-dir-path local-dirpath

In the above example:

-- r option stands for recursive. It will be useful while copying directories

-- user is the user name of the remote host

-- remote-host.com is the IP address/DNS of the remote host

ssh

Sometimes, we need to execute a command on the remote host. Obviously, we can log in to that server and execute the command there, but what if we want to capture the output of that command and use it on the local machine? In such a scenario, we can instruct SSH to execute the command on the remote host using the syntax given below:

$ ssh user@remote-host.com [COMMAND]

For instance, the command given below executes the ls command on the remote host:

$ ssh user@remote-host.com ls

rsync rsync is a remote as well as local file-copying tool.

The rsync utility is used to synchronise the files and directories from one location to another in an effective way.

To synchronise directories on the local host, execute the rsync command as follows:

$ rsync -zvr src-dir target-dir

In the above example:

-- z option stands for ‘enable compression’

-- v option stands for ‘verbose mode’

-- r option stands for ‘recursive mode’

To synchronise the remote directory, we have to provide an IP address and user name for that host. For instance, the following command synchronises the local directory with the remote host: $ rsync -zvr src-dir user@remote-host.com:target-dir

9) cron

We perform many kinds of tasks on a day-to-day basis; for instance, taking backup of important data, checking for updates, and many more. Wouldn’t it be great if we automate these tasks? We can achieve this using cron. We can write cron jobs, which will be scheduled periodically. This section provides practical examples of cron.

To list all available cron jobs, execute the command given below:

$ crontab -l

If any cron job is configured, then it’ll be listed here; otherwise, the output will be empty.

Cron jobs are stored in plain text files. To edit those files, we have to use the crontab command. But before that, let us understand the cron job format.

A cron job consists of the following six entries:

M H DOM MON DOW COMMAND

In the above example:

-- M stands for ‘minutes’

-- H stands for ‘hour’

-- DOM stands for ‘day of the month’

-- MON stands for ‘month’

-- DOW stands for ‘day of the week’

-- COMMAND field indicates the command/script to be executed periodically

For instance, to run a job at 5.00 am every week, we can add the following entry:

0 5 * * 1 script.sh

To add the above entry into cron, perform the following steps:

-- Enter the crontab -e command in the terminal and follow the on-screen instructions:

$ crontab -e -- Add the cron job entry and save the file. 0 5 * * 1 script.sh That’s it; and cron will schedule this job at the right time.

10) System monitoring

GNU/Linux provides many utilities to monitor the system. We can monitor memory usage, disk usage, CPU usage and so on. This section discusses some of the popular utilities that can be used to monitor memory and disk usage.

free

GNU/Linux provides the free command to check memory usage. It displays the total amount of free and used physical and swap memory in the system, as well as the buffers and caches used by the kernel. Shown below is a sample output of the free command:

$ free total used free shared buff/cache available Mem: 8117768 1267836 3718996 153112 3130936 6393176 Swap: 2097148 0 2097148

In the above output:

-- ‘total’ stands for the total installed memory on current system

-- ‘used’ stands for the used memory. It is calculated as follows: [total - (free + buffers + cache) memory]

-- ‘free’ stands for unused memory

-- ‘shared’ stands for the shared memory used by tmpfs -- ‘buffers’ stands for the memory used by kernel buffers -- ‘cache’ stands for the memory used by the page cache and slabs

-- ‘buff/cache’ stands for the sum of the buffers and cache memory

-- ‘available’ stands for an estimation of how much memory is available for starting new applications, without swapping.

du

As the name suggests, the du command is used to calculate disk usage. It summarises disk usage of the set of files/directories.

To calculate the size of the directory, execute the command given below:

$ du -sh DIR-PATH

In the above example:

-- option s is used to display only a total for each argument -- option h is used to show the output in human readable format (K for KB, M for MB and so on)

df

As the name suggests, the df command is used to get information about free disk space. It reports the file system’s disk space usage. If no file name is given, the space available on all currently mounted file systems is shown:

$ df -h

In the above example:

-- option h is used to show output in a human readable format (K for KB, M for MB and so on)

In this article, we have discussed some of the popular GNU/Linux utilities briefly. Mastering these utilities will take your knowledge to the next level. To know more about each utility, do refer to the official documentation.