How to install Graylog on Ubuntu 16.04

Graylog Logo
Graylog

Graylog (formerly known as Graylog2) is an open-source log management tool, helps you to collect, index and analyze any machine logs centrally. This guide focuses on installing Graylog on Ubuntu 16.04, as well as other essential components that make Graylog a powerful log management tool.

Components:

1. MongoDB – Acts as a database, stores the configurations and meta information.

2. Elasticsearch – It stores the log messages and offers a searching facility. It is recommended to allocate more memory and use SAS or SAN disks for Elasticsearch nodes. Here, where all your searching happens.

3. Graylog server  – Log parser, it collects the logs from various inputs and provides built-in Web Interface for managing the logs.

Prerequisites:

As you know, Elasticsearch is a java based application. Install either openJDK or Oracle JDK on your machine to proceed further.

PS: I choose to install Oracle JDK.

Verify the Java version.

$ java -version

java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Install Elasticsearch:

Elasticsearch is one of the main component which requires Graylog to run, acts as a search server, offers a real-time distributed search and analytics with the RESTful web interface. Elasticsearch stores all the logs sent by the Graylog server and displays the messages whenever user request over the built-in web interface.

This guide covers configuration settings that are required for Graylog; you can also take a look at Install Elasticsearch on CentOS 7 / Ubuntu 14.10 / Linux Mint 17.1 for detailed instruction.

Let’s install the Elasticsearch. First download and install GPG signing key.

$ wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Configure Eleasticsearch repository by running below command.

$ echo "deb https://packages.elastic.co/elasticsearch/2.x/debian stable main" | sudo tee -a /etc/apt/sources.list.d/elasticsearch.list

Update repository cache and install Elasticsearch.

$ sudo apt-get update && sudo apt-get install -y elasticsearch

Make Elasticsearch to start automatically on the system startup.

$  sudo systemctl enable elasticsearch

The only important thing is to set a cluster name as “graylog“, edit the configuration file of Elasticsearch and update it accordingly.

$ sudo nano /etc/elasticsearch/elasticsearch.yml

cluster.name: graylog

PS: cluster.name in elasticsearh.yml should match the value of elasticsearch_cluster_name in server.conf of graylog

Disable dynamic scripts to avoid remote execution, by adding the following lines to the server.conf.

script.inline: false
script.indexed: false
script.file: false

Restart the Elasticsearch service to read the new configurations.

$ sudo service elasticsearch restart

Wait at least a minute to let the Elasticsearch get fully restarted. Elastisearch should be now listening on 9200 for processing HTTP request, use a CURL to check the response.

Ensure that cluster name shows as “graylog

$ curl -X GET http://localhost:9200

{
  "name" : "Marvin Flumm",
  "cluster_name" : "graylog",
  "version" : {
    "number" : "2.3.3",
    "build_hash" : "218bdf10790eef486ff2c41a3df5cfa32dadcfde",
    "build_timestamp" : "2016-05-17T15:40:04Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

Optional:  Test the health of Elasticsearch cluster, make sure the output yields the cluster status as “green

$ curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

{
  "cluster_name" : "graylog",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 1,
  "active_shards" : 1,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Install MongoDB 3.2:

Download and install the latest MongoDB from the official website. Import public key on the terminal to begin.

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927

Add mongodb repository by creating the /etc/apt/sources.list.d/mongodb-org.list file using following command.

$ echo "deb http://repo.mongodb.org/apt/debian wheezy/mongodb-org/3.2 main" | sudo tee /etc/apt/sources.list.d/mongodb-org.list

Install MongoDB using the following command.

$ sudo apt-get update && sudo apt-get install -y mongodb-org

Start the MongoDB and enable it on the system start-up.

$ sudo systemctl start mongod
$ sudo systemctl enable mongod

Install Graylog 2.0.3:

Graylog-server accepts and processes the log messages, displays it for the requests that come from graylog web interface.

Download and Install graylog 2.x repository.

$ wget https://packages.graylog2.org/repo/packages/graylog-2.0-repository_latest.deb
$ sudo dpkg -i graylog-2.0-repository_latest.deb

Install https support and update the repository cache.

$ sudo apt-get install -y apt-transport-https
$ sudo apt-get update

Install Graylog server using the following command.

$ sudo apt-get install -y graylog-server

Edit the server.conf file to begin the graylog configuration.

$ sudo nano /etc/graylog/server/server.conf

You must set a secret to secure the user passwords, use the pwgen command to the same.

$ pwgen -N 1 -s 96

OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

If you get an error like “pwgen: command not found“, install pwgen using the following command.

$ sudo apt-get install pwgen

Place the secret like below.

password_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

Next is to set a hash (sha256) password for the root user (not to be confused with the system user, root user of graylog is admin). You will need this password to login into the web interface, admin’s password can’t be changed using web interface; you must edit this variable to set.

Replace “yourpassword” with the choice of yours.

# echo -n yourpassword | sha256sum

e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

Place the hash password.

root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

You can setup email address admin user.

root_email = "itzgeek.web@gmail.com"

Set timezone of root (admin) user.

root_timezone = UTC

Graylog server will try to find the Elasticsearch nodes automatically by using multicast mode. But when it comes to larger network, it is recommended to use unicast mode which is best suited one for production.

Add the following entry to graylog server.conf file, replace ipaddress with your ipaddress. You can add multiple hosts with comma separated.

elasticsearch_discovery_zen_ping_unicast_hosts = ipaddress:9300

Set only one master node by defining the below variable, the default setting is true.

If you add any second Graylog node, set it to false to make the node as a slave. Master node does some periodic tasks that slave nodes won’t perform.

is_master = true

Set the number of log messages to keep per index; it is recommended to have several smaller indices instead of larger ones.

elasticsearch_max_docs_per_index = 20000000

The following parameter defines to have a total number of indices, if this number is reached old index will be deleted.

elasticsearch_max_number_of_indices = 20

Shards setting rely on the number of nodes in the particular Elasticsearch cluster, if you have only one node, set it as 1.

elasticsearch_shards = 1

This the number of replicas for your indices, if you have only one node in Elasticsearch cluster; set it as 0.

elasticsearch_replicas = 0

Install Graylog web interface:

From the version 2.x,  no more extra web interface component, the web interface is being served directly by Graylog server.

Configure Graylog web interface by editing the server.conf file.

$ sudo nano /etc/graylog/server/server.conf

Modify the below entries to let Graylog Web Interface to connect to the Graylog server.

rest_listen_uri = http://your-server-ip:12900/
web_listen_uri = http://your-server-ip:9000/

Restart Graylog service.

$ sudo systemctl daemon-reload
$ sudo systemctl restart graylog-server

Make Graylog server to start automatically on system startup.

$ sudo systemctl enable graylog-server

You can check out the server startup logs; it will be useful for you to troubleshoot Graylog in case of any issue.

$ sudo tailf /var/log/graylog-server/server.log

On the successful start of graylog-server, you should get the following message in the log file.

2016-07-01T08:21:41.538Z INFO  [ServerBootstrap] Graylog server up and running.

Access Graylog web interface:

The web interface will now be listening on port 9000, point your browser to http://ip-add-ress:9000.

Login with username “admin” and the password you configured at root_password_sha2 on server.conf.

Install Graylog on Ubuntu 16.04 - Login Screen
Install Graylog on Ubuntu 16.04 – Login Screen

Once you logged in, you would see the getting started page.

Install Graylog on Ubuntu 16.04 - Getting Started
Install Graylog on Ubuntu 16.04 – Getting Started

Click on System/Overview to know the status of Graylog server.

Install Graylog on Ubuntu 16.04 -System Overview
Install Graylog on Ubuntu 16.04 -System Overview

Configure Graylog Inputs:

Graylog inputs need to be configured to receive the logs from the external source, i.e., Syslog or any logging system.
Click System –> Inputs –>  select Syslog UDP and then click Launch new input. Fill with the values in the screen like below.

Install Graylog on Ubuntu 16.04 - Creating Syslog Input
Install Graylog on Ubuntu 16.04 – Creating Syslog Input

Once you have created the inputs, configure rsyslog or forward any system logs to yourip-address:1514
Following screenshot shows the logs received by Graylog (Graylog console –> Search).

Install Graylog on Ubuntu 16.04 - Syslog Messages
Install Graylog on Ubuntu 16.04 – Syslog Messages

That’s All! you have successfully installed Graylog 2.0.3 on Ubuntu 16.04.

POSTS YOU MAY LIKE -:)

Share This Post

  • 1326855252

    如果有一天,我潇洒死去,请记得,我来过这里!

  • 蒂欧娜

    您的博客拥有旺盛的生命力!!

  • EDGAR SÁNCHEZ OLAYA

    How do you configure the “rsyslog or forward any system logs to your–ip-address:1514” ?
    Thanks.

  • Phil Pearce

    With the upgrade to 2.1 using the upgrade instructions off the grayloc docs. I get a load of chef errors. Do you get similar?

Shares