Live plotting
Andrew Davison describes how to generate dynamically updating plots that can be viewed directly in a ‘dumb’ terminal window.
Andrew Davison describes how to generate dynamically updating plots, which can be viewed directly in a ‘dumb’ terminal window.
One popular choice for logging into Linux machines on Windows is PUTTY. Admittedly, it only acts as a dumb terminal, but a command line interface is perfectly good for coding, especially when a fancy GUI would slow down the network connection. One possible drawback would seem to be the inability to generate live plots of machine activity using the beautiful graphics in Gnuplot (see John Lane’s article in LXF244 for an introduction),
A live plot is a graph that’s periodically updated to reflect changes to its input data. That’s well within
Gnuplot’s skillset, and can be coded using a while loop and calls to replot and pause . However, this tutorial describes a variation of that approach, where a shell script wakes up Gnuplot accordingly when it’s time to update the plot.
The restriction to a dumb terminal turns out to be no problem for Gnuplot either – it can generate graphs for a wide range of devices, including dumb ones. The resulting graphics are a little primitive, so the script described here also produces PNG files, and an animated GIF. The GIF acts as a simple record of all the changes to the plot during the script’s execution.
See Figure 1 (top right) for a summary of the approach. The shell script runs in a loop, collecting data and generating a data file suitable for Gnuplot. When it’s time for plotting, the script sends an integer to the
Gnuplot script via a pipe. The script is waiting for integer input, then reads in the data file, and generates new text-based and PNG graphs. Then it suspends again, waiting for another integer.
When the shell script is interrupted with Ctrl+c, it terminates after first converting the PNGS into the animated GIF. Gnuplot has a similar capability, but it can’t be easily combined with plotting to multiple output terminals.
The first half of this article gives two examples of this approach; the first has Gnuplot generate a live histogram showing changes to three types of network connection. The second is a slightly more complex livepoints plot of total network activity, which displays the six most recent data points only.
This technique has two (or perhaps even three) drawbacks. The first is that no data history is maintained, since the data file is overwritten. The second is that the generated animation can easily become very large, which may be a problem on small systems. The third problem, which is more a matter of taste, is that an animation can be quite hard to examine and/or analyse. It certainly looks nice, but is it an effective way to log data?
So the second half of this article addresses these problems by outlining two other ways to draw live plots: using its multiplot feature, and using 3D graphics.
A live histogram
Figure 2 (page 79) shows a histogram of established, listening, and waiting connections on a Raspberry Pi. The script is terminated when the user types Ctrl+c (visible underneath the graph), and converts the generated PNGS into the animated GIF before exiting. The histogram is updated every second, which is reflected in the timestamp above the graph.
The shell script we’ve created for this (hist.sh) begins with a few constants: Datafnm=”hist.dat” # data file PLOTFNM=”HIST.GP” # Gnuplot script let DELAY=1 # seconds CONNTYPES=(“ESTABLISHED” “LISTEN” “TIME_ WAIT”)
Most of the script’s actual work is done by the
gendata() function:
gendata()
{ let count=0 while true ; do echo -e -n “” > $1 # create / clear the data file # add data lines for ct in ${conntypes[*]}; do num=$(netstat -ant | grep $ct | wc -l) echo -e $ct”\t”$num >> $1 # add one line done echo $count # wake up gnuplot let count++ sleep $DELAY done
} The data file (hist.dat) is periodically overwritten with three lines of the form:
ESTABLISHED 3
LISTEN 4
TIME_WAIT 0
Gnuplot is woken by the script outputting an integer, which is sent through a pipe set up with: gendata $DATAFNM | gnuplot -persist -e “datafnm=’${datafnm}’” $PLOTFNM
The -persist option ensures that the plot remains on the screen after Gnuplot exits. The -e option is used to pass the datafnm variable into the script so that it knows the name of the data file.
The final part of the shell script is the function called when Ctrl+c is trapped: trap ctrl_c INT ctrl_c()
{ fnms=”animate\_[0-9][0-9][0-9][0-9].png” count=$(ls -1 $fnms | wc -l) echo -n “Converting $count PNGS to an animated GIF...” convert -scale 75% -quiet $fnms animate.gif rm -f $fnms # remove the PNGS echo “done” exit
}
The PNG files are assumed to be called animate_ XXXX.PNG, where XXXX is a four-digit number starting at 0. The Imagemagick convert command scales the images and constructs the animated GIF, after which the PNGS are deleted.
Meanwhile, in Gnuplot…
The Gnuplot script (hist.gp) sets up the dumb terminal output and suitable plotting attributes, and then enters a loop which keeps plotting the histogram. set terminal dumb size 40 20 set boxwidth 0.3 set border 3 # only left and bottom axes shown set xtics nomirror; set ytics nomirror set tics out set xtics scale 0; set ytics scale 0.5 set yrange [0:*] set title “No. of Connected Types” set key outside center top count = @readint # wait for an integer message
# count is used to label the PNG files while (count >= 0) { # -ve signals the shell’s end plot datafnm using 2:xtic(1) notitle with boxes, \ ‘’ using 0:2:2 title @readdate with labels center call “savepng.gp” count set terminal dumb; set output # revert to dumb count = @readint # wait for shell script
}
Gnuplot live plots are usually controlled by having a loop pause for some fixed amount of time between each replot. This script instead uses a call to the
readint macro, which causes the script to wait until an integer is read from the pipe coming from the shell script. The macro uses Gnuplot’s system() function to utilize a bit of Bash:
readint = “int(system(\”read n; echo \$n\”))”
Gnuplot doesn’t currently support subroutines, but it is possible to call other scripts. The savepng.gp script switches the terminal to PNG and calls replot :
set terminal png set output sprintf(“animate_%04d.png”, @ARG1) replot savepng.gp uses Gnuplot’s ARG notation to read in the count argument passed from hist.gp.
A live-points plot
The second example illustrates a slightly more complex form of data updating, since the graph only shows the last six collected data points. Example output is shown in Figure 3 (page 80, bottom left).
The X axis shows the time in minutes and seconds, and is updated so that the points appear to move gradually left, to disappear off the left hand side of the graph. The separation of code between the shell and
Gnuplot scripts means that the data-updating task is handled by the shell script, while Gnuplot only needs to consider how to plot the data. The data file (points.dat) consists of Unix-epoch time values and the number of network connections:
1554886041 132
: #many more lines
1554886046 129
The points.sh script sets up a pipe to Gnuplot in the same way as the earlier hist.sh. It only differs in how its
gendata() function collects data and updates the data file, like this:
gendata()
{ echo -e -n “” > $1 # create / clear the data file while true ; do cutline $1 # my function; see below ts=$(date +%s) num=$(netstat -an | wc -l) echo -e $ts”\t”$num >> $1 # add a line of data echo 0 # wake up gnuplot sleep $DELAY done
} The Bash magic in cutline() removes an old data point by employing sed to delete the first line of the file: cutline()
{ len=$(wc -l < $1) if [ “$len” -ge “$MAXPTS” ] then sed -i “1d” $1