ELK in one (Vagrant) box


In this blog post we introduce a Vagrant box to easily create configurable and reproducible development environments for ELK (Elasticsearch, Logastash and Kibana). At Comperio, we mainly use this box for query log analysis using the ELK stack.
In case you don’t know, Vagrant is a free and open-source software that combines VirtualBox (a virtualization software) with configuration management softwares such as Puppet and Chef.

ELK stack up and running in two commands

$ git clone https://github.com/comperiosearch/vagrant-elk-box.git
$ vagrant up

By cloning this github repo and then typing “vagrant up”, you will be installing elasticsearch, logstash, kibana and nginx (the latter used to serve kibana).

Elasticsearch will be running on port 9200, as usual, which is forwarded to the host machine. As for Kibana, it will be served on port 5601 (also accessible from the host OS).

How does it work?
As mentioned above, Vagrant is a wrapper around VirtualBox and some configuration management software. In our box, we use pure shell scripting and Puppet to configure the ELK stack.
There are two essential configuration files in this box: Vagrantfile and the Puppet manifest default.pp.
Vagrantfile includes the settings of the virtual box such as operating system, memory size, number of CPUs, forwarded ports, etc…

Vagrantfile also includes a shell script that installs, among other things, the official Puppet modules for elasticsearch and logstash. By using that shell script we stay away from git submodules which were used in another Vagrant image we made earlier for elasticsearch.

In the Puppet manifest, default.pp, we define what version of elasticsearch to install and make sure that it is running as a service.

We do the same for logstash and additionally link the default logstash configuration file to this file under /Vagrant/confs/logstash which is shared with the host OS. Finally, we install nginx and Kibana, and configure Kibana to run on port 5601 (by linking the nginx conf file to this file in the Vagrant directory also).

13 response to: «ELK in one (Vagrant) box»

  1. September 16, 2014 at 23:19 | Permalink

    Hello Murhaf,

    I have just released an ELK stack environment based on vagrant/virtualbox.
    If you want to give it a try it is available here : https://github.com/frntn/vagrant-elk-clientserver

    Feedbacks are welcome too (https://twitter.com/@frntn)

  2. September 24, 2014 at 14:33 | Permalink

    Nice! thanks for sharing :)

  3. October 19, 2014 at 22:13 | Permalink

    I am getting a “connection failed” when trying to connect to Kibana on

    Possibility #1: Your elasticsearch server is down or unreachable
    This can be caused by a network outage, or a failure of the Elasticsearch process. If you have recently run a query that required a terms facet to be executed it is possible the process has run out of memory and stopped. Be sure to check your Elasticsearch logs for any sign of memory pressure.
    Possibility #2: You are running Elasticsearch 1.4 or higher
    Elasticsearch 1.4 ships with a security setting that prevents Kibana from connecting. You will need to set http.cors.allow-origin in your elasticsearch.yml to the correct protocol, hostname, and port (if not 80) that your access Kibana from. Note that if you are running Kibana in a sub-url, you should exclude the sub-url path and only include the protocol, hostname and port. For example, http://mycompany.com:8080, not http://mycompany.com:8080/kibana.

  4. October 21, 2014 at 11:09 | Permalink

    It is likely that the elasticsearch service is not running, did you check it?

  5. October 21, 2014 at 12:27 | Permalink

    Yes, I think the elasticsearch is not running.

    Don’t you want ES to be running?



  6. October 21, 2014 at 14:54 | Permalink

    Thanks Anastasios!

    The issue is now fixed, please see: https://github.com/comperiosearch/vagrant-elk-box/issues/2

  7. November 6, 2014 at 10:34 | Permalink

    The box ‘hashicorp/precise64′ could not be found.

  8. November 14, 2014 at 14:08 | Permalink

    Hey Nik.
    Most likely this is because you are running a version of Vagrant older than 1.5*. You get version 1.4-something if you install it through apt-get on Ubuntu.

    Please try downloading the latest version from http://www.vagrantup.com and try again.
    And if you want you can run
    vagrant init ubuntu/trusty64
    instead, to get a newer version of Ubuntu.

  9. January 28, 2015 at 23:14 | Permalink

    I’m kind of new to the ELK stack, so I might be missing something basic here, but I’m having a hard time getting logstash to actually read any data and put it in elasticsearch.

    I have modified the logstash.conf file to remove the index and template that you set: https://gist.github.com/anonymous/eace13f91498dbf49970

    I was not too sure about the index, but I had to remove the path to the template otherwise I kept getting an error in the logstash log.

    I put a bunch of log files in /vagrant/project/logs as indicated in the logstash conf.

    Even though I have fully reloaded the VM, it seems like none of the data in my log files is getting pulled in. For example, when I check this URL– http://localhost:9200/_search?pretty — I see 1 hit in index kibana-int. I don’t see anything from the log files I put in the log dir above.

    Lastly, it looks like logstash is configured to reload whenever a configuration file changes, but I’m having trouble observing that behavior. I can see that the PID changes of the java process that’s running logstash, but I don’t see any output in /var/logs/logstash/logstash.*

    Even if I deliberately put an error in my logstash conf, I see no output in the logs. If I reload the vagrant VM, I see the error I would expect. This seems inconsistent and I’m not sure what’s going on.

    Thanks very much for this post.

  10. January 30, 2015 at 10:11 | Permalink

    Seems like the permissions for kibana are not correct/have changed.
    When connecting to http://localhost:5601 I receive a 403 Forbidden from nginx.

  11. February 12, 2015 at 18:37 | Permalink


    I got a conflict during the puppet run due to a duplicated declaration of the class Java:

    Error: Duplicate declaration: Class[Java] is already declared in file /vagrant/manifests/default.pp:26; cannot redeclare at /etc/puppet/modules/elasticsearch/manifests/init.pp:331 on node precise64

    I commented line 26 out in /vagrant/manifests/default.pp and it worked for me after i triggered a new puupet run: puppet apply /vagrant/manifests/default.pp

    Thanks for the great vagrant box, it is really easy!!

  12. February 16, 2015 at 20:06 | Permalink

    The project has now been updated to work with Elasticsearch 1.4, Logstash 1.4 and Kibana 4 (RC),

    The install reports timeout errors starting Kibana, but it works just fine.

    Keep on ELKing :)

  13. April 15, 2015 at 10:02 | Permalink

    hi what are the credentials.

    Additionally, i can ping from ELK guest to mac os host. but vice versa is not working. network adapter is nat.

Leave a response

XHTML: These tags are allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">

Couldn't connect to server: Connection timed out (110)