Installing Graylog2 on Ubuntu with Apache2

Graylog2 is an extremely interesting project for logging with MongoDB, so we wanted to give it a try. While there are some installation instructions out there, we had to do some digging around for getting the setup just the way we wanted. So here's our own step-by-step installation instruction:

Prerequisites: We're assuming you already have MongoDB, Apache2, and Java up and running on your Linux box (Ubuntu in our specific case, but at least Debian should be very similar).

First you'll need to set up the server (we're using the lastest version in this example, but this will obviously change in the future) - this is very straight forward:

• wget https://github.com/downloads/Graylog2/graylog2-server/graylog2-server-0.9.5p1.tar.gz
• tar xvfz graylog2-server-0.9.5p1.tar.gz
• cd graylog2-server-0.9.5p1/
• sudo cp graylog2.conf.example /etc/graylog2.conf
• Edit the settings according to your MongoDB installation: sudo pico /etc/graylog2.conf
• cd bin/
• sudo ./graylog2ctl start
• ps aux | grep gray

Once this is working as expected, you might want to automatically start the Graylog2 server. Assuming you moved it to /usr/local/graylog2-server/:

• sudo touch /etc/init.d/graylog2-server
• sudo pico /etc/init.d/graylog2-server
  

  #! /bin/sh
  ### BEGIN INIT INFO
  # Provides: Starts Graylog2 Server
  # Required-Start:
  # Required-Stop:
  # Default-Start: 2 3 4 5
  # Default-Stop: 0 1 6
  # Short-Description: Graylog2 Server
  # Description: Server aggregating the logs
  ### END INIT INFO
  # Author: Philipp Krenn
  
  # Aktionen
  case "$1" in
  start)
  cd /usr/local/graylog2-server/bin/
  ./graylog2ctl start
  ;;
  stop)
  cd /usr/local/graylog2-server/bin/
  ./graylog2ctl stop
  ;;
  restart)
  cd /usr/local/graylog2-server/bin/
  ./graylog2ctl restart
  ;;
  esac
  
  exit 0

• sudo update-rc.d graylog2-server defaults

Once this is done we can install the Ruby on Rails based frontend. We wanted to install it into a subdirectory of Apache2, which made it a little trickier:

• Get the required dependencies: sudo apt-get update && sudo apt-get install ruby1.8 rubygems rake make libopenssl-ruby ruby-dev build-essential git-core
• cd ~
• Get the Graylog2 frontend: wget https://github.com/downloads/Graylog2/graylog2-web-interface/graylog2-web-interface-0.9.5p2.tar.gz
• tar xvfz graylog2-web-interface-0.9.5p2.tar.gz
• Get the latest Ruby Gems to bundle the installation: wget http://production.cf.rubygems.org/rubygems/rubygems-1.8.5.tgz
• tar xvfz rubygems-1.8.5.tgz
• cd rubygems-1.8.5/
• sudo ruby setup.rb
• sudo gem update
• sudo gem install bundler
• cd ~/graylog2-web-interface-0.9.5p2/
• bundle install
• bundle show
• Apache2's base directory is /var/www/, so let's create /var/rails/ and put the Rails files there. We'll then link the Apache2 directory to the Rails application.
• sudo mkdir /var/rails/
• sudo mkdir /var/rails/graylog2
• sudo cp -R ./* /var/rails/graylog2/
• Install Apache2's Passenger module, which integrates Rails applications.
  
• sudo apt-get install libapache2-mod-passenger
• sudo gem install passenger
• sudo passenger-install-apache2-module
  
• The previous step ran the module's installer, which will do quite some work for you. Additionally it will tell you which dependencies are not yet met, in our case we had to add the following packages: sudo apt-get install libcurl4-openssl-dev libssl-dev zlib1g-dev apache2-prefork-dev libapr1-dev libaprutil1-dev
• Run the tool again and execute the steps it tells you to, except for the last one wanting you to change some configurations: sudo passenger-install-apache2-module
• Instead open up the following file and make sure only the given content is available there: sudo pico /etc/apache2/mods-available/passenger.conf
  

  <ifmodule c="">
  PassengerRoot /usr
  
  PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-3.0.7
  PassengerRuby /usr/bin/ruby1.8
  </ifmodule>

• Do the same for another configuration file: sudo pico /etc/apache2/mods-available/passenger.load
  

  LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-3.0.7/ext/apache2/mod_passenger.so

• Now add your subdirectory to your Apache2's configuration (inside the VHost past): sudo pico /etc/apache2/sites-available/default
  

  RailsBaseURI /graylog2
  

• Link the public/ directory of your Rails application to the one you set up in Apache2's configuration: sudo ln -s /var/rails/graylog2/public/ /var/www/graylog2
• Change the configuration, this must the be same as the server configuration: sudo pico /var/www/graylog/config/mongoid.yml
  
• Finally restart Apache2: sudo /etc/init.d/apache2 restart

That's it, the Graylog2 frontend should now be available in the /graylog2 subdirectory.


UPDATE: As noted by Lennart (https://twitter.com/#!/_lennart/status/88330653246038016) you need to manually create your indexes: In /var/rails/graylog2 run the command sudo rake db:mongoid:create_indexes RAILS_ENV=production and you're good to go.

Logging MongoDB Queries

Why do you need to log queries?

• In case you want to know how long it takes to execute a specific query.
• To see which is the slowest query.
  
• If you want to see which queries are actually being run (in case you're using a layer of abstraction like Morphia this can be a pretty interesting question)
• ...

Using MongoDB's built in profiling tool it's very simple to achieve this:

1. You can enable and disable profiling on a per database level (being disabled by default).
2. Connect to the database on the shell / CMD (assuming you're using the port 8082): mongo localhost:8082/erpel_test
3. Activate profiling for all queries, not just slow ones: db.setProfilingLevel(2);
4. Exit the shell / CMD and run some queries.
5. Your database should now have a new collection called system.profile, showing the raw queries and how long it took to execute them.

For additional profiling options see http://www.mongodb.org/display/DOCS/Database+Profiler.