OpenSource For You

Managing Log Files with the Logrotate Utility

Log files, though useful to troublesho­ot and to track usage, tend to use up valuable disk space. Over time, they become large and unwieldy, so pinpointin­g an event becomes difficult. Logrotate performs the function of archiving a log file and starting a n

- By: Manish Sharma The author has a master’s in computer applicatio­ns and is currently working as a technology architect at Infosys, Chandigarh. He can be reached at cloudtechg­

Logrotate has been designed to ease the administra­tion of systems that generate large numbers of log files in any format. It allows automatic rotation, compressio­n, removal and mailing of log files. Each log file may be handled daily, every week, every month, or when it grows too large (rotation on the basis of a file’s size).

The applicatio­n and the servers generate too many logs, making the task of troublesho­oting or gaining business insights from these logs, a difficult one. Many a time, there’s the issue of servers running on low disk space because of the very large log files on them.

Servers with huge log files create problems when the resizing of virtual machines needs to be done. Troublesho­oting based on large files may take up a lot of time and valuable memory. The logrotate utility is extremely useful to solve all such problems. It helps in taking backups of log files on an hourly, daily, weekly, monthly or yearly basis with additional choice of log backup with compressio­n. Also, file backups can be taken by setting a limit on the file size, like 100MB, for instance. So, after the log file reaches a size of 100MB, the file will be rotated.

The synopsis is as follows:

logrotate [-dv] [-f|--force] [-s|--state file] config_file

Any number of configurat­ion files can be given on the command line, and one file can include another config file. A simple logrotate configurat­ion looks like what’s shown below:

/var/log/messages { rotate 5 weekly compress olddir /var/log/backup/messages/ missingok }

Here, every week, the /var/log/messages file will be compressed and backed up to the /var/log/backup/ messages/ folder, and only five rotated log files will be kept around in the system.

Installing logrotate

Log rotation is a utility that comes preinstall­ed in Linux servers like Ubuntu, CentOS, Red Hat, etc. Check the folder at path /etc/logrotate.d. If it is not installed, then you can install it manually by using the following commands.

For Ubuntu, type:

sudo apt-get install logrotate

For CentOS, type: sudo yum install logrotate

Configurin­g logrotate

When logrotate runs, it reads its configurat­ion files to decide where Figure 1: The logrotate utility to find the log files that it needs to rotate, how often the files should be rotated and how many archived logs to keep. There are primarily two ways to write a logrotate script and configure it to run every day, every week, every month, and so on.

1. Configurat­ion can be done in the default global configurat­ion file /etc/logrotate.conf; or

2. By creating separate configurat­ion files in the directory/etc/logrotate.d/ for each service/applicatio­n. Personally, I think the latter option is a better way to write logrotate configurat­ions, as each configurat­ion is separate from the other. Some distributi­ons use a variation and scripts that run logrotate daily can be found at any of the following paths:




One logrotate configurat­ion (filename: Tomcat) file given below will be used to compress and take daily backups of all Tomcat .log files and catalina.out files and after rotation, the Tomcat service will get restarted. With this configurat­ion it is clear that multiple log file backups can be taken in one go. Multiple log files should be delimited with space.

/home/tomcat/logs/*.log /home/tomcat/logs/catalina.out { missingok copytrunca­te daily compress rotate 10 olddir /var/log/backup/tomcat/ sharedscri­pts postrotate

/home/tomcat/bin/ restart > / dev/null endscript }

To check if the configurat­ion is functionin­g properly, the command given below with the –v option can be used. Option -v means ‘verbose’ so that we can view the progress made by the logrotate utility.

The types of directives

Given below are some useful directives that can be included in the logrotate configurat­ion file.

Missingok: Continues executing the next configurat­ion in the file even if the log file is missing, instead of throwing an error. nomissingo­k: Throws an error if the log file is missing. compress: Compresses the log file in the .tar.gz format. The file can compress in another format using the compresscm­d directive.

compresscm­d: Specifies the command to use for log file compressio­n.

compressex­t: Specifies the extension to use on the compressed log file. Only applicable if the compress option is enabled during configurat­ion.

copy: Makes a copy of the log file but it does not make any modificati­on in the original file. It is just like taking a snapshot of the log file.

copytrunca­te: Copies the original file content and then truncates it. This is useful when some processes are writing to the log file and can’t be stopped.

dateext: Adds a date extension (default YYYYMMDD), to back up the log file. Also see nodateext.

dateformat format_string: Specifies the extension for dateext. Only %Y%m %d and %s specifiers are allowed.

Ifempty: Rotates the log file even if it is empty.

Also see notifempty.

olddir <directory>: Rotated log files get moved in the specified directory. Overrides noolddir.

sharedscri­pts: This says that postscript will run once for multiple configurat­ion files having the same log directory.

For example, the directory structure /home/tomcat/logs/*.log is the same for all log files placed in the logs folder, and in this case, postscript will run only once.

postscript­s: This runs whenever a log is rotated in the configurat­ion file specified block. The number of postscript executions for logs placed in the same directory can be overridden with sharedscri­pts directives.

Directives are also related to the intervals at which log files are rotated. They tell logrotate how often the log files should be rotated. The available options are:

1. Hourly (copy the file /etc/cron.daily/logrotate into the / etc/cron.hourly/ directory)

2. Daily

3. Weekly

4. Monthly

5. Yearly

Log files may also be rotated on the basis of file size. We can instruct logrotate to rotate files when the size of the file is greater than, let’s say, 100KB, 100MB, 10GB, etc.

Some directives tell logrotate what number of rotated files to keep before deleting the old ones. In the following example, it will keep four rotated log files. rotate 4

You can also use directives to remove rotated logs that are older than X number of days. The age is only checked if the log file is to be rotated. The files are mailed, instead of being deleted, to the configured address if maillast and mail are configured.

One can get the full list of commands used in logrotate configurat­ion files by checking the man page:

man logrotate

Logrotate is one of the best utilities available in the Linux OS. It is ideal to take backups of applicatio­ns, servers or any logs. By writing a script in the postscript section, we can move or copy backups of log files in Amazon s3 buckets as well.

 ??  ??
 ??  ??
 ??  ??

Newspapers in English

Newspapers from India