How To Install Graylog 3.0 on CentOS 7 / RHEL 7

9

Graylog is an open-source log management tool which helps you to collect, index and analyze any machine logs centrally. This guide will help you to install Graylog on CentOS 7 / RHEL 7.

Components

  1. MongoDB – Being a database to store the configurations and meta information.
  2. Elasticsearch – It stores the log messages received from the Graylog server and provides a facility to search them whenever required. Elasticsearch is a resource monger as it does indexing of data, so allocate more memory and use SAS or SAN disks.
  3. Graylog server – This does the parsing of logs that are coming from various inputs and provides built-in Web Interface to handle those logs.

Prerequisites

Make sure your system is configured with EPEL repository.

Install the required packages for Graylog installation.

yum -y install wget pwgen

Elasticsearch requires Java to be installed on the machine. So, install either OpenJDK or Oracle JDK.

yum install -y java-1.8.0-openjdk-headless

Verify the Java version.

java -version

Output:

openjdk version "1.8.0_191"
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

Install Elasticsearch

Elasticsearch is one of the important components in the Graylog setup. It stores the data coming from Graylog input and displays the messages whenever a user request over the Graylog built-in web interface.

Elasticseach is mainly used here to index the data/logs and provide the searching functionality when the Graylog web interface request for any information.

This guide covers the basic configuration required for Graylog.

Import the GPG signing key before the installation.

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create a repository file to get the latest version of Elasticsearch package from the official repository.

vi /etc/yum.repos.d/elasticsearch.repo

Add the below content to above file.

[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Now, install the Elasticsearch package using yum command.

yum install -y elasticsearch

Reload the systemctl daemon and enable Elasticsearch to start automatically on the system startup.

systemctl daemon-reload
systemctl enable elasticsearch

To make Elasticsearch work with Graylog setup, we need to set the cluster name to graylog. Edit the elasticsearch.yml file.

vi /etc/elasticsearch/elasticsearch.yml

Update it like shown below.

cluster.name: graylog

Restart the Elasticsearch.

systemctl restart elasticsearch

Give a minute to let the Elasticsearch get fully restarted. Elastisearch should be now listening to 9200 for processing HTTP requests. Use the CURL command to check the response.

curl -X GET http://localhost:9200

Cluster name should be graylog.

{
  "name" : "DF8QK3-",
  "cluster_name" : "graylog",
  "cluster_uuid" : "_wAgUfN9RJeQ0npCKBswVA",
  "version" : {
    "number" : "6.6.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "a9861f4",
    "build_date" : "2019-01-24T11:27:09.439740Z",
    "build_snapshot" : false,
    "lucene_version" : "7.6.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Perform a health check of Elasticsearch cluster.

curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

Make sure the cluster status comes as green.

{
  "cluster_name" : "graylog",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

Install MongoDB

MongoDB acts as a database to store the configurations and meta information. We will use MongoDB’s official repository to get the latest version of MongoDB.

vi /etc/yum.repos.d/mongodb-org-4.0.repo

Add repository information to the above file.

[mongodb-org-4.0]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.0/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc

Install the community edition of MongoDB using the following command.

yum install -y mongodb-org

Start the MongoDB service and enable it to the system start-up.

systemctl start mongod
systemctl enable mongod

Install Graylog

Graylog server accepts and processes the log messages coming from the various inputs, displays data to requests that come from the graylog web interface.

Download and Install graylog 3.x repository.

rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.0-repository_latest.rpm

Install the Graylog server using the following command.

yum -y install graylog-server

Edit the server.conf file to begin the graylog configuration.

vi /etc/graylog/server/server.conf

Use the following command to create a secret

pwgen -N 1 -s 96

Output:

OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

Place the secret like below in server.conf

password_secret = OH9wXpsNZVBA8R5vJQSnkhTB1qDOjCxAh3aE3LvXddtfDlZlKYEyGS24BJAiIxI0sbSTSPovTTnhLkkrUvhSSxodTlzDi5gP

Set the hash password for the root user, i.e., admin of graylog. You would need this password to login into the graylog web.

If you ever want to change/reset the forgotten password of admin, you can edit/update the server.conf with a hashed password.

Generate a hashed password using the below command. Replace yourpassword with the choice of yours.

echo -n yourpassword | sha256sum

Output:

e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

Place the hash password.

root_password_sha2 = e3c652f0ba0b4801205814f8b6bc49672c4c74e25b497770bb89b22cdeb4e951

You can set up an email address for the admin user.

root_email = "[email protected]"

Set timezone of root (admin) user.

root_timezone = UTC

Set only one master node by defining the below variable, the default setting is true.

If this server is your second Graylog node in your environment, set it to false to make this node as a slave. Master node does some periodic tasks that slave nodes won’t perform.

is_master = true

Set the number of log messages to keep per index. It is recommended to have several smaller indices instead of larger ones.

elasticsearch_max_docs_per_index = 20000000

The following parameter defines to have a total number of indices. If this number is reached, the old index will be deleted.

elasticsearch_max_number_of_indices = 20

Shards setting rely on the number of nodes in the particular Elasticsearch cluster. If you have only one node, set it as 1.

elasticsearch_shards = 1

This defines the number of replicas for your indices. If you have only one node in the Elasticsearch cluster, set it as 0.

elasticsearch_replicas = 0

Install Graylog web interface

From the Graylog version 2.x, the web interface is served directly by the Graylog server. Enable the Graylog web interface by editing the server.conf file.

vi /etc/graylog/server/server.conf

Modify the entries to let Graylog Web Interface to connect to the Graylog server. Replace 192.168.1.10 with your system IP address.

http_bind_address = 192.168.1.10:9000

Restart Graylog service.

systemctl daemon-reload
systemctl restart graylog-server

Make Graylog server to start automatically on system startup.

systemctl enable graylog-server

You can check out the server startup logs; it will be useful for you to troubleshoot Graylog for any issues.

tailf /var/log/graylog-server/server.log

If everything goes well, you should see below message in the server.log file.

2019-02-25T21:00:23.157+05:30 INFO  [ServerBootstrap] Graylog server up and running.

Firewall

In CentOS 7 / RHEL 7, firewall rules are set to block most of the traffic coming from the external machines by default. You would need to add allow rules for Graylog to work correctly.

firewall-cmd --permanent --add-port=9000/tcp

Reload firewalld to take an effect of the new rules.

firewall-cmd --reload

Access Graylog web interface

The web interface will now be listening on port 9000. Point your browser to.

http://ip.add.re.ss:9000

Login with username admin and the password you configured at root_password_sha2 on server.conf.

Install Graylog 3.0 on CentOS 7 - Gralog Login Screen
Install Graylog 3.0 on CentOS 7 – Gralog Login Screen

Once you logged in, you would see the getting started page.

Install Graylog 3.0 on CentOS 7 - Graylog Dashboard
Install Graylog 3.0 on CentOS 7 – Graylog Dashboard

Click on System >> Overview to know the status of Graylog server.

Install Graylog 3.0 on CentOS 7 - System Overview
Install Graylog 3.0 on CentOS 7 – System Overview

Create Graylog Inputs

Graylog input needs to be configured to receive the logs from external sources, i.e., syslog or any logging system. For this example, we will set up an input for receiving logs from syslog on port number UDP 514.

But, there is a problem. If you start to try an input on one of the privileged ports, any TCP/UDP port number below 1024, you would see permission denied error in Graylog logs.

To resolve the issue, we will create an input to start on port 1514, and we will redirect the traffic comes to 514 to 1514 with firewall rules.

Click System >> Inputs >> select Syslog UDP and then click Launch new input.

Install Graylog 3.0 on CentOS 7 - Syslog Input
Install Graylog 3.0 on CentOS 7 – Syslog Input

Fill the values shown below.

Node: Select your Graylog Node

Title: Name your input

Bind address: 0.0.0.0 (Leave the default one)

Port: 1514

Install Graylog 3.0 on CentOS 7 - Create Syslog UDP Input
Install Graylog 3.0 on CentOS 7 – Create Syslog UDP Input

Ensure the created Syslog input is in running state.

Install Graylog 3.0 on CentOS 7 - Greylog Input Status
Install Graylog 3.0 on CentOS 7 – Graylog Input Status

Configure Rsyslog

Once you have created the inputs, configure Rsyslog or forward any system logs to your Graylog server.

Edit the Rsyslog configuration file.

vi /etc/rsyslog.conf

Typical Rsyslog configuration will look like below to send logs on UDP 514.

If you do not wish to use UDP 514 and want to send logs directly to UDP 1514 of Graylog, then you can change the port number in the Rsyslog configuration file
*.info;mail.none;authpriv.none;cron.none @192.168.1.10:514

Restart the Rsyslog service to send logs.

systemctl restart rsyslog

Firewall

Configure the firewall based on Rsyslog configuration you did on the previous step.

Port 514

Execute the below commands on the Graylog server to redirect the traffic that comes on port UDP 514 to UDP 1514 of Graylog input.

firewall-cmd --add-masquerade --permanent
firewall-cmd --add-forward-port=port=514:proto=udp:toport=1514 --permanent
firewall-cmd --reload

Port 1514

Execute the below commands on the Graylog server to allow port number UDP 1514 on the firewall to receive logs from Rsyslog.

firewall-cmd --permanent --add-port=1514/udp
firewall-cmd --reload

View Logs using Graylog

Wait for a few minutes. You should start receiving log messages from the client machine when the event is generated.

Following screenshot shows the logs received by Graylog (Graylog console >> Search).

Install Graylog 3.0 on CentOS 7 - View Syslog Messages using Graylog
Install Graylog 3.0 on CentOS 7 – View Syslog Messages using Graylog

Conclusion

You have successfully installed Graylog 3.0 on CentOS 7 / RHEL 7 and configured Graylog to receive Rsyslog logs from external sources. As a further read, you can try configuring Nginx or Apache as a reverse proxy and set up HTTPS for the Graylog web interface.

You might also like