Im Logfile finden wir eine Readahead-Warnmeldung mit dem Hinweis, dass die readahead-Einstellungen für das Block-Gerät auf dem die MongoDB gespeichert wurde nicht optimal zum Speichern der NoSQL-Datenbank MongoDB geeigent ist.

 2015-12-23T11:05:52.630+0100 [initandlisten] ** WARNING: Readahead for /var/lib/mongodb is set to 4096KB
 2015-12-23T11:05:52.630+0100 [initandlisten] **          We suggest setting it to 256KB (512 sectors) or less
 2015-12-23T11:05:52.630+0100 [initandlisten] **          http://dochub.mongodb.org/core/readahead

Zunächst suchen wir, auf welchem Blockdevice das Verzeichnis /var/lib/mongodb liegt.

 # mount | grep /var/lib/mongodb

/dev/vdc1 on /var/lib/mongodb type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

Fragen wir die Einstellungen unserer Blockdevices ab erhalten wir zunächst:

 # blockdev --report

RO    RA   SSZ   BSZ   StartSec            Size   Device
rw  8192   512  4096          0     10485760000   /dev/vda
rw  8192   512   512       2048       524288000   /dev/vda1
rw  8192   512  4096    1026048      9960423424   /dev/vda2
rw  8192   512  4096          0     21474836480   /dev/vdb
rw  8192   512   512       2048     21473787904   /dev/vdb1
rw  8192   512  4096          0     21474836480   /dev/vdc
rw  8192   512   512       2048     21473787904   /dev/vdc1
rw  8192   512  4096          0      1073741824   /dev/dm-0
rw  8192   512   512          0      6731857920   /dev/dm-1
rw  8192   512   512          0      2147483648   /dev/dm-2

Wir setzen nunmehr den Wert RA auf die empfohlene Größe von 256KB (512 sectors) für das Blockdevice /dev/vdc1 auf dem unsere MongoDB bespeichert wird.

 # blockdev --setra 256 /dev/vdc1

Fragen wir erneut die Einstellungen für /dev/vdc1 sehen wir, dass der Parameter RA nun den Wert von 256 hat.

 # blockdev --report | grep /dev/vdc1

rw   256   512   512       2048     21473787904   /dev/vdc1

Wenn abschließend starten wir nun den MongoDB-Daemon einmal durch, damit unsere Änderung auch wirksam werden kann.

 # systemctl restart mongod.service

Die Warnmeldung ist nunmehr verschwunden.

 # less /var/log/mongodb/mongod.log

2015-12-23T12:11:50.221+0100 [initandlisten] MongoDB starting : pid=16609 port=27017 dbpath=/var/lib/mongodb 64-bit host=vml000117.dmz.nausch.org
2015-12-23T12:11:50.221+0100 [initandlisten] db version v2.6.11
2015-12-23T12:11:50.221+0100 [initandlisten] git version: nogitversion
2015-12-23T12:11:50.221+0100 [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2015-12-23T12:11:50.221+0100 [initandlisten] build info: Linux buildvm-08.phx2.fedoraproject.org 4.1.4-100.fc21.x86_64 #1 SMP Tue Aug 4 03:25:05 UTC 2015 x86_64 BOOST_LIB_VERSION=1_53
2015-12-23T12:11:50.221+0100 [initandlisten] allocator: tcmalloc
2015-12-23T12:11:50.221+0100 [initandlisten] options: { command: [ "run" ], config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", unixDomainSocket: { pathPrefix: "/var/run/mongodb" } }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, storage: { dbPath: "/var/lib/mongodb" }, systemLog: { destination: "file", path: "/var/log/mongodb/mongod.log", quiet: true } }
2015-12-23T12:11:50.235+0100 [initandlisten] journal dir=/var/lib/mongodb/journal
2015-12-23T12:11:50.235+0100 [initandlisten] recover : no journal files present, no recovery needed
2015-12-23T12:11:51.759+0100 [initandlisten] preallocateIsFaster=true 13.14
2015-12-23T12:11:52.660+0100 [initandlisten] waiting for connections on port 27017

Die eben gemachte Konfigurationsberichtigung wird aber bei einem Reboot des Servers wieder verworfen. Wir müssen also dafür sorgen, dass der readahead-Wert bei einem Neustart des Systems auf den gewünschten Wert 256 gesetzt wird. Hierzu legen wir uns ein kleines Bash-Script an.

 #  vim /usr/local/bin/setra256

/usr/local/bin/setra256

#!/bin/bash
# Django : 2015-12-28
# Setzen des readahead Wertes auf 256KB (512 Sektoren)
/usr/sbin/blockdev --setra 256 /dev/vdc1

Nun müssen wir nur noch dafür sorgen, dass dieses Script bei einem Neustart gestartet wird. Dazu legen wir uns einen eigenen „kleinen Service“ an, in dem wir im Verzeichnis /etc/systemd/system/ eine systemd-Startscript anlegen.

 # vim /etc/systemd/system/setra256.service

/etc/systemd/system/setra256.service

[Unit]
Description=Initialize hardware monitoring sensors
After=syslog.target network.target
Before=mongod.service
 
[Service]
Type=oneshot
ExecStart=/usr/local/bin/setra256
 
[Install]
WantedBy=multi-user.target

Zu guter letzt führen wir noch einen Reload des Systemd-Daemon durch und wir haben künftig immer den richtig gesetzten readahead von 256 gesetzt.

 # systemctl daemon-reload

Damit der Daemon beim Hochfahren unseres Servers automatisch gestartet wird, nutzen wir folgenden Befehl.

 # systemctl enable mongod.service

Created symlink from /etc/systemd/system/multi-user.target.wants/mongod.service to /usr/lib/systemd/system/mongod.service.

Wollen wir wissen, ob die Autostartfunktion bereits gesetzt ist, verwenden wir diesen Aufruf.

 # systemctl is-enabled mongod.service

enabled

Wie bereits kurz erwähnt wollen wir unsere MongoDB absichern, indem wir einen Datenbakbenutzer mit Passwort anlegen. Dazu verbinden wir uns erst einmal mit der MongoDB Shell mongo.

 # mongo

MongoDB shell version: 2.6.11
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
> 

Zunächst wechseln wir zur Datenbank admin mit folgendem Aufruf.

> use admin

switched to db admin
>

Anschließend legen wir den Datenbankadministrator admin mit dem Passwort 5y510953rv3r53cr37 an.

> db.createUser({user:"admin",pwd:"5y510953rv3r53cr37",roles:[{role:"root",db:"admin"}]})

Successfully added user: {
        "user" : "admin",
        "roles" : [
                {
                        "role" : "root",
                        "db" : "admin"
                }
        ]
}
> 

Für den Zugriff von graylog auf die NoSQL-Datenbang MongoDB erstellen wir uns nun noch einen separaten Nutzer graylog-user mit dem zugehörigen Passwort 7h3FBI15n07ar0ckb4and. Bevor wir diesen Datenbank-Account anlegen, erstellen und wechseln wir noch zur Datenbank graylog.

> use graylog

switched to db graylog

Nun können wir auch unseren User graylog-user anlegen.

> db.createUser({user:"graylog-user",pwd:"7h3FBI15n07ar0ckb4and",roles:["readWrite"]})

Successfully added user: { "user" : "graylog-user", "roles" : [ "readWrite" ] }
> 

Wir haben die nötigen Definitionen erfolgreich zu Ende gebracht und nun können wir die MongoDB Shell mongo wieder verlassen.

> exit

bye

Zum Testen, ob unser Datenbanknutzer graylog-user sich auch erfolgreich mit der NoSQL-Datenbank graylog verbinden kann, melden wir uns mit dessen Daten an und fragen die vorhandenen Datenbanken ab.

 # mongo -u "graylog-user" -p "7h3FBI15n07ar0ckb4and" 127.0.0.1:27017/graylog

MongoDB shell version: 2.6.11
connecting to: 127.0.0.1:27017/test
> 

Die Tabellen der gewählten Datenbank kann man sich mit dem Befehl show collections anzeigen lassen.

> show collections

alarmcallbackconfigurations
alarmcallbackhistory
alerts
cluster_config
cluster_events
collectors
content_packs
dashboards
dead_letters
grok_patterns
index_failures
index_ranges
inputs
ldap_settings
nodes
notifications
roles
sessions
streamrules
streams
system.indexes
system_messages
users
>

Da der Zugriff klappte, können wir die Datenbankverbindung wieder beenden.

> exit

bye

Geben wir ein falsches Passwort ein, wird natürlich der Zugang verwehrt.

 # mongo -u "graylog-user" -p "7h3FBI15n07ar0ckb4and" 127.0.0.1:27017/graylog

MongoDB shell version: 2.6.11
connecting to: 127.0.0.1:27017/test
2015-12-23T13:08:02.381+0100 Error: 18 { ok: 0.0, errmsg: "auth failed", code: 18 } at src/mongo/shell/db.js:1292
exception: login failed

Zum Abschluss unterbinden wir nun noch den Passwortlosen Zugang zur MongoDB. Der Parameter auth im Abschnitt General options der Konfigurationsdatei /etc/mongod.conf

 # vim /etc/mongod.conf

/etc/mongod.conf

##                                       
### Basic Defaults                       
##                                       
 
# Comma separated list of ip addresses to listen on (all local ips by default)
bind_ip = 127.0.0.1                                                           
 
# Specify port number (27017 by default)
#port = 27017                           
 
# Fork server process (false by default)
fork = true                             
 
# Full path to pidfile (if not set, no pidfile is created)
pidfilepath = /var/run/mongodb/mongod.pid                 
 
# Log file to send write to instead of stdout - has to be a file, not directory
logpath = /var/log/mongodb/mongod.log                                          
 
# Alternative directory for UNIX domain sockets (defaults to /tmp)
unixSocketPrefix = /var/run/mongodb                               
 
# Directory for datafiles (defaults to /data/db/)
dbpath = /var/lib/mongodb                        
 
# Enable/Disable journaling (journaling is on by default for 64 bit)
#journal = true                                                     
#nojournal = true                                                   
 
 
 
##
### General options
##                 
 
# Be more verbose (include multiple times for more verbosity e.g. -vvvvv) (v by default)
#verbose = v                                                                            
 
# Max number of simultaneous connections (1000000 by default)
#maxConns = 1000000                                          
 
# Log to system's syslog facility instead of file or stdout (false by default)
#syslog = true                                                                
 
# Syslog facility used for monogdb syslog message (user by defautl)
#syslogFacility = user                                             
 
# Append to logpath instead of over-writing (false by default)
#logappend = true                                             
 
# Desired format for timestamps in log messages (One of ctime, iso8601-utc or iso8601-local) (iso8601-local by default)
#timeStampFormat = arg                                                                                                 
 
# Private key for cluster authentication
#keyFile = arg                          
 
# Set a configurable parameter
#setParameter = arg           
 
# Enable http interface (false by default)
#httpinterface = true                     
 
# Authentication mode used for cluster authentication. Alternatives are (keyFile|sendKeyFile|sendX509|x509) (keyFile by default)
#clusterAuthMode = arg                                                                                                          
 
# Disable listening on unix sockets (false by default)
#nounixsocket = true                                  
 
# Run with/without security (without by default)
#auth = true                                    
#noauth = true                                  
# Django : 2015-12-23 - Passwortlose logins unterbinden
# default: #auth = true                                
#          #noauth = true                              
auth = true                                            
 
# Enable IPv6 support (disabled by default)
#ipv6 = true                               
 
# Allow JSONP access via http (has security implications) (false by default)
#jsonp = true                                                               
 
# Turn on simple rest api (false by default)
#rest = true                                
 
# Value of slow for profile and console log (100 by default)
#slowms = 100                                               
 
# 0=off 1=slow, 2=all (0 by default)
#profile = 0                        
 
# Periodically show cpu and iowait utilization (false by default)
#cpu = true                                                      
 
# Print some diagnostic system information (false by default)
#sysinfo = true                                              
 
# Each database will be stored in a separate directory (false by default)
#directoryperdb = true                                                   
 
# Don't retry any index builds that were interrupted by shutdown (false by default)
#noIndexBuildRetry = true                                                          
 
# Disable data file preallocation - will often hurt performance (false by default)
#noprealloc = true                                                                
 
# .ns file size (in MB) for new databases (16 MB by default)
#nssize = 16                                                
 
# Limits each database to a certain number of files (8 default)
#quota                                                         
 
# Number of files allowed per db, implies --quota (8 by default)
#quotaFiles = 8                                                 
 
# Use a smaller default file size (false by default)
#smallfiles = true                                  
 
# Seconds between disk syncs (0=never, but not recommended) (60 by default)
#syncdelay = 60                                                            
 
# Upgrade db if needed (false by default)
#upgrade = true                          
 
# Run repair on all dbs (false by default)
#repair = true                            
 
# Root directory for repair files (defaults to dbpath)
#repairpath = arg                                     
 
# Disable scripting engine (false by default)
#noscripting = true                          
 
# Do not allow table scans (false by default)
#notablescan = true                          
 
# Journal diagnostic options (0 by default)
#journalOptions = 0                        
 
# How often to group/batch commit (ms) (100 or 30 by default)
#journalCommitInterval = 100                                 
 
 
 
##
### Replication options
##                     
 
# Size to use (in MB) for replication op log (default 5% of disk space - i.e. large is good)
#oplogSize = arg                                                                            
 
 
 
##
### Master/slave options (old; use replica sets instead)
##                                                      
 
# Master mode
#master = true
 
# Slave mode
#slave = true
 
# When slave: specify master as <server:port>
#source = arg                                
 
# When slave: specify a single database to replicate
#only = arg                                         
 
# Specify delay (in seconds) to be used when applying master ops to slave
#slavedelay = arg                                                        
 
# Automatically resync if slave data is stale
#autoresync = true                           
 
 
 
##
### Replica set options
##                     
 
# Arg is <setname>[/<optionalseedhostlist>]
#replSet = arg
 
# Specify index prefetching behavior (if secondary) [none|_id_only|all] (all by default)
#replIndexPrefetch = all
 
 
 
##
### Sharding options
##
 
# Declare this is a config db of a cluster (default port 27019; default dir /data/configdb) (false by default)
#configsvr = true
 
# Declare this is a shard db of a cluster (default port 27018)  (false by default)
#shardsvr = true
 
 
 
##
### SSL options
##
 
# Use ssl on configured ports
#sslOnNormalPorts = true
 
# Set the SSL operation mode (disabled|allowSSL|preferSSL|requireSSL)
# sslMode = arg
 
# PEM file for ssl
#sslPEMKeyFile = arg
 
# PEM file password
#sslPEMKeyPassword = arg
 
# Key file for internal SSL authentication
#sslClusterFile = arg
 
# Internal authentication key file password
#sslClusterPassword = arg
 
# Certificate Authority file for SSL
#sslCAFile = arg
 
# Certificate Revocation List file for SSL
#sslCRLFile = arg
 
# Allow client to connect without presenting a certificate
#sslWeakCertificateValidation = true
 
# Allow server certificates to provide non-matching hostnames
#sslAllowInvalidHostnames = true
 
# Allow connections to servers with invalid certificates
#sslAllowInvalidCertificates = true
 
# Activate FIPS 140-2 mode at startup
#sslFIPSMode = true

Nun werden wir den MongoDB-Daemon einaml noch durchstarten, damit die durchgeführten Änderungen auch aktiv werden.

 # systemctl restart mongod.service

Ein Anmeldeversuch ohne Benutzerkennung samt Passwort ist ab sofort nicht mehr möglich.

In diesem Abschnitt werden wir nun den Suchserver elasticsearch, der auch die eigentlichen Nutztdaten von graylog vorhalten wird, konfigurieren.

In der Dokumentation von graylog im Abschnitt Architectural considerations finden wir folgenden Hinweis:

Diesem Vorschlag wollen wir nun in die Tat umsetzen. Beim Blockdevice haben wir in unserem Konfigurationsbeispiel bereits ein RAID aus schnellen Server-SSD's verwendet. Nun werden wir dem elasticsearch-Daemon noch mehr an RAM zugestehen. Hierzu werden wir nun den Parameter vm.max_map_count anpassen. Zunächst aber fragen wird den aktuellen Wert von vm.max_map_count erst einmal ab.

 # sysctl vm.max_map_count

vm.max_map_count = 65530

In der Dokumentation von elasticsearch zum virtuellen Speicherverbrauch wird empfohlen, den großzügigen Wert von 262144 zu setzen. Während der Laufzeit des Systems können wir die Änderung mit folgendem Befehl vornehmen.

 # sysctl -w vm.max_map_count=262144

vm.max_map_count = 262144

Damit die Werteanpassung resetfest, also bei jedem Neustart des Servers auf den gewünschten Wert gesetzt wird, bearbeiten wir die Datei /etc/sysctl.conf.

 # vim /etc/sysctl.conf

/etc/sysctl.conf

# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
#
# Django : 2015-12-23 - den virtuellen Speicherverbrauch von elasticsearch anpassen
vm.max_map_count=262144

Laut der Installationsdokumentation von graylog beschränkt sich die Konfiguration von elasticsearch auf das Setzen des cluster name. Den Node Name setzen wir im dem Zuge auf den Hoastname unseres Syslog-Servers.

Da aber in der Dokumentation Setup Configuration werden wir noch den Parameter mlockall auf true setzen.

 # vim /etc/elasticsearch/elasticsearch.yml

/etc/elasticsearch/elasticsearch.yml

##################### Elasticsearch Configuration Example #####################                                  
 
# This file contains an overview of various configuration settings,
# targeted at operations staff. Application developers should      
# consult the guide at <http://elasticsearch.org/guide>.           
#                                                                  
# The installation procedure is covered at                         
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup.html>.
#                                                                                
# Elasticsearch comes with reasonable defaults for most settings,                
# so you can try it out without bothering with configuration.                    
#                                                                                
# Most of the time, these defaults are just fine for running a production        
# cluster. If you're fine-tuning your cluster, or wondering about the            
# effect of certain configuration option, please _do ask_ on the                 
# mailing list or IRC channel [http://elasticsearch.org/community].              
 
# Any element in the configuration can be replaced with environment variables
# by placing them in ${...} notation. For example:                           
#                                                                            
#node.rack: ${RACK_ENV_VAR}                                                  
 
# For information on supported formats and syntax for the config file, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/setup-configuration.html>
 
 
################################### Cluster ###################################
 
# Cluster name identifies your cluster for auto-discovery. If you're running
# multiple clusters on the same network, make sure you're using unique names.
#                                                                            
# Django : 2015-12-28                                                        
# default: #cluster.name: elasticsearch                                      
cluster.name: graylog                                                        
 
 
#################################### Node #####################################
 
# Node names are generated dynamically on startup, so you're relieved
# from configuring them manually. You can tie this node to a specific name:
#                                                                          
# Django : 2015-12-28                                                      
# default: #node.name: "Franz Kafka"                                       
node.name: vml000117                                                       
 
# Every node can be configured to allow or deny being eligible as the master,
# and to allow or deny to store the data.                                    
#                                                                            
# Allow this node to be eligible as a master node (enabled by default):      
#                                                                            
#node.master: true                                                           
#                                                                            
# Allow this node to store data (enabled by default):                        
#                                                                            
#node.data: true                                                             
 
# You can exploit these settings to design advanced cluster topologies.
#                                                                      
# 1. You want this node to never become a master node, only to hold data.
#    This will be the "workhorse" of your cluster.                       
#                                                                        
#node.master: false                                                      
#node.data: true                                                         
#                                                                        
# 2. You want this node to only serve as a master: to not store any data and
#    to have free resources. This will be the "coordinator" of your cluster.
#                                                                           
#node.master: true                                                          
#node.data: false                                                           
#                                                                           
# 3. You want this node to be neither master nor data node, but             
#    to act as a "search load balancer" (fetching data from nodes,          
#    aggregating results, etc.)                                             
#                                                                           
#node.master: false                                                         
#node.data: false                                                           
 
# Use the Cluster Health API [http://localhost:9200/_cluster/health], the
# Node Info API [http://localhost:9200/_nodes] or GUI tools              
# such as <http://www.elasticsearch.org/overview/marvel/>,               
# <http://github.com/karmi/elasticsearch-paramedic>,                     
# <http://github.com/lukas-vlcek/bigdesk> and                            
# <http://mobz.github.com/elasticsearch-head> to inspect the cluster state.
 
# A node can have generic attributes associated with it, which can later be used
# for customized shard allocation filtering, or allocation awareness. An attribute
# is a simple key value pair, similar to node.key: value, here is an example:     
#                                                                                 
#node.rack: rack314                                                               
 
# By default, multiple nodes are allowed to start from the same installation location
# to disable it, set the following:                                                  
#node.max_local_storage_nodes: 1                                                     
 
 
#################################### Index ####################################
 
# You can set a number of options (such as shard/replica options, mapping
# or analyzer definitions, translog settings, ...) for indices globally, 
# in this file.                                                          
#                                                                        
# Note, that it makes more sense to configure index settings specifically for
# a certain index, either when creating it or by using the index templates API.
#                                                                              
# See <http://elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules.html> and
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/indices-create-index.html> 
# for more information.                                                                         
 
# Set the number of shards (splits) of an index (5 by default):
#                                                              
#index.number_of_shards: 5                                     
 
# Set the number of replicas (additional copies) of an index (1 by default):
#                                                                           
#index.number_of_replicas: 1                                                
 
# Note, that for development on a local machine, with small indices, it usually
# makes sense to "disable" the distributed features:                           
#                                                                              
#index.number_of_shards: 1                                                     
#index.number_of_replicas: 0                                                   
 
# These settings directly affect the performance of index and search operations
# in your cluster. Assuming you have enough machines to hold shards and        
# replicas, the rule of thumb is:                                              
#                                                                              
# 1. Having more *shards* enhances the _indexing_ performance and allows to    
#    _distribute_ a big index across machines.                                 
# 2. Having more *replicas* enhances the _search_ performance and improves the 
#    cluster _availability_.                                                   
#                                                                              
# The "number_of_shards" is a one-time setting for an index.                   
#                                                                              
# The "number_of_replicas" can be increased or decreased anytime,              
# by using the Index Update Settings API.                                      
#                                                                              
# Elasticsearch takes care about load balancing, relocating, gathering the     
# results from nodes, etc. Experiment with different settings to fine-tune     
# your setup.                                                                  
 
# Use the Index Status API (<http://localhost:9200/A/_status>) to inspect
# the index status.                                                      
 
 
#################################### Paths ####################################
 
# Path to directory containing configuration (this file and logging.yml):
#                                                                        
#path.conf: /path/to/conf                                                
 
# Path to directory where to store index data allocated for this node.
#                                                                     
#path.data: /path/to/data                                             
#                                                                     
# Can optionally include more than one location, causing data to be striped across
# the locations (a la RAID 0) on a file level, favouring locations with most free 
# space on creation. For example:                                                 
#                                                                                 
#path.data: /path/to/data1,/path/to/data2                                         
 
# Path to temporary files:
#                         
#path.work: /path/to/work 
 
# Path to log files:
#                   
#path.logs: /path/to/logs
 
# Path to where plugins are installed:
#                                     
#path.plugins: /path/to/plugins       
 
 
#################################### Plugin ###################################
 
# If a plugin listed here is not installed for current node, the node will not start.
#                                                                                    
#plugin.mandatory: mapper-attachments,lang-groovy                                    
 
 
################################### Memory ####################################
 
# Elasticsearch performs poorly when JVM starts swapping: you should ensure that
# it _never_ swaps.                                                             
#                                                                               
# Set this property to true to lock the memory:                                 
#                                                                               
# Django : 2015-12-28                                                           
# default: unset                                                                
bootstrap.mlockall: true                                                        
 
 
# Make sure that the ES_MIN_MEM and ES_MAX_MEM environment variables are set
# to the same value, and that the machine has enough memory to allocate     
# for Elasticsearch, leaving enough memory for the operating system itself. 
#                                                                           
# You should also make sure that the Elasticsearch process is allowed to lock
# the memory, eg. by using `ulimit -l unlimited`.                            
 
 
############################## Network And HTTP ###############################
 
# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).                                                            
 
# Set the bind address specifically (IPv4 or IPv6):
#                                                  
#network.bind_host: 192.168.0.1                    
 
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.  
#                                                                           
#network.publish_host: 192.168.0.1                                          
 
# Set both 'bind_host' and 'publish_host':
#                                         
#network.host: 192.168.0.1                
 
# Set a custom port for the node to node communication (9300 by default):
#                                                                        
#transport.tcp.port: 9300                                                
 
# Enable compression for all communication between nodes (disabled by default):
#                                                                              
#transport.tcp.compress: true                                                  
 
# Set a custom port to listen for HTTP traffic:
#                                              
#http.port: 9200                               
 
# Set a custom allowed content length:
#                                     
#http.max_content_length: 100mb       
 
# Disable HTTP completely:
#                         
#http.enabled: false      
 
 
################################### Gateway ###################################
 
# The gateway allows for persisting the cluster state between full cluster
# restarts. Every change to the state (such as adding an index) will be stored
# in the gateway, and when the cluster starts up for the first time,          
# it will read its state from the gateway.                                    
 
# There are several types of gateway implementations. For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html>.
 
# The default gateway type is the "local" gateway (recommended):
#                                                               
#gateway.type: local                                            
 
# Settings below control how and when to start the initial recovery process on
# a full cluster restart (to reuse as much local data as possible when using shared
# gateway).                                                                        
 
# Allow recovery process after N nodes in a cluster are up:
#                                                          
#gateway.recover_after_nodes: 1                            
 
# Set the timeout to initiate the recovery process, once the N nodes
# from previous setting are up (accepts time value):                
#                                                                   
#gateway.recover_after_time: 5m                                     
 
# Set how many nodes are expected in this cluster. Once these N nodes
# are up (and recover_after_nodes is met), begin recovery process immediately
# (without waiting for recover_after_time to expire):                        
#                                                                            
#gateway.expected_nodes: 2                                                   
 
 
############################# Recovery Throttling #############################
 
# These settings allow to control the process of shards allocation between
# nodes during initial recovery, replica allocation, rebalancing,         
# or when adding and removing nodes.                                      
 
# Set the number of concurrent recoveries happening on a node:
#                                                             
# 1. During the initial recovery                              
#                                                             
#cluster.routing.allocation.node_initial_primaries_recoveries: 4
#                                                               
# 2. During adding/removing nodes, rebalancing, etc             
#                                                               
#cluster.routing.allocation.node_concurrent_recoveries: 2       
 
# Set to throttle throughput when recovering (eg. 100mb, by default 20mb):
#                                                                         
#indices.recovery.max_bytes_per_sec: 20mb                                 
 
# Set to limit the number of open concurrent streams when
# recovering a shard from a peer:                        
#                                                        
#indices.recovery.concurrent_streams: 5                  
 
 
################################## Discovery ##################################
 
# Discovery infrastructure ensures nodes can be found within a cluster
# and master node is elected. Multicast discovery is the default.     
 
# Set to ensure a node sees N other master eligible nodes to be considered
# operational within the cluster. This should be set to a quorum/majority of 
# the master-eligible nodes in the cluster.                                  
#                                                                            
#discovery.zen.minimum_master_nodes: 1                                       
 
# Set the time to wait for ping responses from other nodes when discovering.
# Set this option to a higher value on a slow or congested network          
# to minimize discovery failures:                                           
#                                                                           
#discovery.zen.ping.timeout: 3s                                             
 
# For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html>
 
# Unicast discovery allows to explicitly control which nodes will be used
# to discover the cluster. It can be used when multicast is not present, 
# or to restrict the cluster communication-wise.                         
#                                                                        
# 1. Disable multicast discovery (enabled by default):                   
#                                                                        
#discovery.zen.ping.multicast.enabled: false                             
# Django : 2015-12-28                                                    
# default: "enabled by default"                                          
discovery.zen.ping.multicast.enabled: false                              
 
#
# 2. Configure an initial list of master nodes in the cluster
#    to perform discovery when new nodes (master or data) are started:
#                                                                     
#discovery.zen.ping.unicast.hosts: ["host1", "host2:port"]            
# Django : 2015-12-28                                                 
# default: unset                                                      
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]                  
 
 
# EC2 discovery allows to use AWS EC2 API in order to perform discovery.
#
# You have to install the cloud-aws plugin for enabling the EC2 discovery.
#
# For more information, see
# <http://elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-ec2.html>
#
# See <http://elasticsearch.org/tutorials/elasticsearch-on-ec2/>
# for a step-by-step tutorial.
 
# GCE discovery allows to use Google Compute Engine API in order to perform discovery.
#
# You have to install the cloud-gce plugin for enabling the GCE discovery.
#
# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-gce>.
 
# Azure discovery allows to use Azure API in order to perform discovery.
#
# You have to install the cloud-azure plugin for enabling the Azure discovery.
#
# For more information, see <https://github.com/elasticsearch/elasticsearch-cloud-azure>.
 
################################## Slow Log ##################################
 
# Shard level query and fetch threshold logging.
 
#index.search.slowlog.threshold.query.warn: 10s
#index.search.slowlog.threshold.query.info: 5s
#index.search.slowlog.threshold.query.debug: 2s
#index.search.slowlog.threshold.query.trace: 500ms
 
#index.search.slowlog.threshold.fetch.warn: 1s
#index.search.slowlog.threshold.fetch.info: 800ms
#index.search.slowlog.threshold.fetch.debug: 500ms
#index.search.slowlog.threshold.fetch.trace: 200ms
 
#index.indexing.slowlog.threshold.index.warn: 10s
#index.indexing.slowlog.threshold.index.info: 5s
#index.indexing.slowlog.threshold.index.debug: 2s
#index.indexing.slowlog.threshold.index.trace: 500ms
 
################################## GC Logging ################################
 
#monitor.jvm.gc.young.warn: 1000ms
#monitor.jvm.gc.young.info: 700ms
#monitor.jvm.gc.young.debug: 400ms
 
#monitor.jvm.gc.old.warn: 10s
#monitor.jvm.gc.old.info: 5s
#monitor.jvm.gc.old.debug: 2s
 
################################## Security ################################
 
# Uncomment if you want to enable JSONP as a valid return transport on the
# http server. With this enabled, it may pose a security risk, so disabling
# it unless you need it is recommended (it is disabled by default).
#
#http.jsonp.enable: true

Damit Parameter bootstrap.mlockall auch richtig gesetzt werden kann, ist noch eine Änderung an dem Systemd-Startscript notwendig! Dieses werden wir nun noch anpassen; hierzu kopieren wir uns nun das entsprechende Script nach /etc/systemd/system/.

 # cp -a /usr/lib/systemd/system/elasticsearch.service /etc/systemd/system/

In dieser Datei setzen wir den Parameter LimitMEMLOCK auf infinity.

 # vim /etc/systemd/system/elasticsearch.service

/etc/systemd/system/elasticsearch.service

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target
 
[Service]
Environment=ES_HOME=/usr/share/elasticsearch
Environment=CONF_DIR=/etc/elasticsearch
Environment=CONF_FILE=/etc/elasticsearch/elasticsearch.yml
Environment=DATA_DIR=/var/lib/elasticsearch
Environment=LOG_DIR=/var/log/elasticsearch
Environment=PID_DIR=/var/run/elasticsearch
EnvironmentFile=-/etc/sysconfig/elasticsearch
 
User=elasticsearch
Group=elasticsearch
 
ExecStart=/usr/share/elasticsearch/bin/elasticsearch \
                                                -Des.pidfile=$PID_DIR/elasticsearch.pid \
                                                -Des.default.path.home=$ES_HOME \
                                                -Des.default.path.logs=$LOG_DIR \
                                                -Des.default.path.data=$DATA_DIR \
                                                -Des.default.config=$CONF_FILE \
                                                -Des.default.path.conf=$CONF_DIR
 
# Connects standard output to /dev/null
StandardOutput=null
 
# Connects standard error to journal
StandardError=journal
 
# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143
 
# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65535
 
# Specifies the maximum number of bytes of memory that may be locked into RAM
# Set to "infinity" if you use the 'bootstrap.mlockall: true' option
# in elasticsearch.yml and 'MAX_LOCKED_MEMORY=unlimited' in /etc/sysconfig/elasticsearch
# Django : 2015-12-28
# default: # LimitMEMLOCK=infinity
LimitMEMLOCK=infinity
 
# Shutdown delay in seconds, before process is tried to be killed with KILL (if configured)
TimeoutStopSec=20
 
[Install]
WantedBy=multi-user.target
 
# Built for elasticsearch-1.7.4 (rpm)

Anschließend informieren wir den systemd über unser „updatesicheres“ Startscript.

 systemctl daemon-reload

Unsere Konfigurationsänderungen werden mit Hilfe von mongo der MongoDB Shell vorgenommen. Dieses JavaScript stellt die Benutzerschnittstelle zum eigentlichen MongoDN-Daemon zur Verfügung. Damit sich der Client mit dem Server auch verbinden kann, muss der Server natürlich laufen; wir starten also nun unsere NoSQL-Datenbank mit folgendem Befehl.

 # systemctl start mongod.service

Den Serverstatus können wir wie folgt abfragen.

 # systemctl status mongod.service

● elasticsearch.service - Elasticsearch
   Loaded: loaded (/etc/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
   Active: active (running) since Mon 2015-12-28 14:20:23 CET; 1min 56s ago
     Docs: http://www.elastic.co
 Main PID: 32221 (java)
   CGroup: /system.slice/elasticsearch.service
           └─32221 /bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupan...

Dec 28 14:20:23 vml000117.dmz.nausch.org systemd[1]: Started Elasticsearch.
Dec 28 14:20:23 vml000117.dmz.nausch.org systemd[1]: Starting Elasticsearch...

 # /var/log/elasticsearch/elasticsearch.log

[2015-12-28 14:20:24,201][INFO ][node                     ] [vml000117] version[1.7.4], pid[32221], build[0d3159b/2015-12-15T16:45:04Z]
[2015-12-28 14:20:24,202][INFO ][node                     ] [vml000117] initializing ...
[2015-12-28 14:20:24,323][INFO ][plugins                  ] [vml000117] loaded [], sites []
[2015-12-28 14:20:24,378][INFO ][env                      ] [vml000117] using [1] data paths, mounts [[/var/lib/elasticsearch (/dev/vdc1)]], net usable_space [24.3gb], net total_space [24.4gb], types [xfs]
[2015-12-28 14:20:29,846][INFO ][node                     ] [vml000117] initialized
[2015-12-28 14:20:29,846][INFO ][node                     ] [vml000117] starting ...
[2015-12-28 14:20:30,282][INFO ][transport                ] [vml000117] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.0.0.117:9300]}
[2015-12-28 14:20:30,352][INFO ][discovery                ] [vml000117] graylog/EdAnadZuTiOjxFR7_Kvdrg
[2015-12-28 14:20:33,462][INFO ][cluster.service          ] [vml000117] new_master [vml000117][EdAnadZuTiOjxFR7_Kvdrg][vml000117.dmz.nausch.org][inet[/10.0.0.117:9300]], reason: zen-disco-join (elected_as_master)
[2015-12-28 14:20:33,508][INFO ][http                     ] [vml000117] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.0.0.117:9200]}
[2015-12-28 14:20:33,508][INFO ][node                     ] [vml000117] started
[2015-12-28 14:20:33,541][INFO ][gateway                  ] [vml000117] recovered [0] indices into cluster_state
[2015-12-28 14:20:35,573][INFO ][cluster.service          ] [vml000117] added {[vml000117][4u6LKkewT-ClHj25SjY5gA][vml000117.dmz.nausch.org][inet[/10.0.0.117:9350]]{client=true, data=false, master=false},}, reason: zen-disco-receive(join from node[[vml000117][4u6LKkewT-ClHj25SjY5gA][vml000117.dmz.nausch.org][inet[/10.0.0.117:9350]]{client=true, data=false, master=false}])
[2015-12-28 14:20:44,296][INFO ][cluster.metadata         ] [vml000117] [graylog_0] creating index, cause [api], templates [graylog-internal], shards [1]/[0], mappings [message]
[2015-12-28 14:20:44,959][INFO ][cluster.metadata         ] [vml000117] [graylog_1] creating index, cause [api], templates [graylog-internal], shards [1]/[0], mappings [message]