Let’s get logging
Before getting some initial set up done, I wanted to quickly look at the work needed on the clients (in this case our many Linux systems). I’ll admit that previously I’ve had misgivings about installing Java specifically just to run a log-forwarding agent (albeit it Logstash is written in JRuby), especially in development shops that didn’t want to touch Java with a bargepole. Luckily, the agent element of Logstash has now evolved into a project called Beats which are lightweight processes written in Go. There are a number of different Beats available. I’m specifically looking at Filebeat which replaces the old logstash-forwarder application. In actual fact, these agents can dump data directly into Elasticsearch if needed (but that means missing out on some of the cool transformational stuff Logstash can do).
Now onto the main event: getting everything up and running. Elastic maintain its own package repositories with the usual distro selection available. I’ll stick to my usual Ubuntu 14.04 setup, which I’m sure is getting boring for many of you! I’m going to install the ELK stack on one VM to begin with and then have some clients send it some logs before looking at some of the ‘fun’ stuff in the next issue.
Take a look at http://bit.ly/ElasticReposSetup which details public keys involved etc. The steps can be summarised as follows: $ wget -qO - https://packages.elastic.co/GPG-KEYelasticsearch | sudo apt-key add - $ echo “deb http://packages.elastic.co/elasticsearch/2.x/ debian stable main” | sudo tee -a /etc/apt/sources.list.d/ elasticsearch-2.x.list $ sudo apt-get update Note: This specifically avoids using add-apt-repository as there is no deb-src repo available. Here I’m using 2.x as the version, following Elastic’s recommendation. I won’t cover installing Java (a prerequisite for installing Elasticsearch itself) which comes with the usual ‘official Oracle packages’ vs OpenJDK dilemma (it should be fine, according to Elastic and was what I used here).
Installing Elasticsearch is as simple as running $ sudo aptget install elasticsearch . This will download about 28MB. Ubuntu will install it as a service as expected if installed via apt which can be kicked off with $ sudo service start elasticsearch .
Jumping over to /var/log/elasticsearch and taking a look at elasticsearch.log should show everything up and running (heavily edited output here)