Elasticsearch and Kibana

• Search Engine Setup and Configuration
  • Installing Elasticsearch 6.x on CentOS
    • Import the Elasticsearch PGP Key
  • Installing from the RPM repository
    • Running Elasticsearch with systemd
    • Checking that Elasticsearch is running
    • Configuring Elasticsearch
  • Installing Kibana 6.x on CentOS
    • Running Kibana with systemd

Search Engine Setup and Configuration

Installing Elasticsearch 6.x on CentOS

Elasticsearch is a distributed, JSON-based search and analytics engine designed for horizontal scalability, maximum reliability, and easy management.

Import the Elasticsearch PGP Key

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Installing from the RPM repository

Create a file called elasticsearch.repo in the /etc/yum.repos.d/ directory and add the following lines:

[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

And your repository is ready for use. You can now install Elasticsearch with one of the following commands:

sudo yum install elasticsearch

Running Elasticsearch with systemd

To configure Elasticsearch to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service

Apparently there is no way to quietly reload the Elasticsearch service after changing the config file - you will be required to stop and restart instead:

sudo systemctl stop elasticsearch.service
sudo systemctl start elasticsearch.service

These commands provide no feedback as to whether Elasticsearch was started successfully or not. Instead, this information will be written in the log files located in /var/log/elasticsearch/.

Checking that Elasticsearch is running

You can test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost:

curl -XGET 'localhost:9200/?pretty'

http://localhost:9200/_cat/indices?v&pretty

Configuring Elasticsearch

Elasticsearch loads its configuration from the /etc/elasticsearch/elasticsearch.yml file by default. Examples:

• cluster.name: e.g. instar-wiki
• node.name e.g. c21
• node.attr.rack: e.g r44
• path.data: /path/to/data
• path.logs: /path/to/logs
• network.host: localhost see config *
• http.port: 9200
• http.cors: enabled: true , allow-origin: /https?://localhost(:[0-9]+)?/, allow-origin: /https?://localhost(:[0-9][0-9][0-9][0-9])?/ * e.g. network.host: 127.0.0.1, 192.168.1.200, 7.114.21.49

The RPM places config files, logs, and the data directory in the appropriate locations for an RPM-based system:

Type	Description	Default Location	Setting
home	Elasticsearch home directory or $ES_HOME	/usr/share/elasticsearch
bin	Binary scripts including elasticsearch to start a node and elasticsearch-plugin to install plugins	/usr/share/elasticsearch/bin
conf	Configuration files including elasticsearch.yml	/etc/elasticsearch	ES_PATH_CONF
conf	Environment variables including heap size, file descriptors.	/etc/sysconfig/elasticsearch
data	The location of the data files of each index / shard allocated on the node. Can hold multiple locations.	/var/lib/elasticsearch	path.data
logs	Log files location.	/var/log/elasticsearch	path.logs
plugins	Plugin files location. Each plugin will be contained in a subdirectory.	/usr/share/elasticsearch/plugins

Installing Kibana 6.x on CentOS

Kibana gives shape to your data and is the extensible user interface for configuring and managing all aspects of the Elastic Stack.

Create a file called kibana.repo in the /etc/yum.repos.d/ directory and add the following lines:

[kibana-6.x]
name=Kibana repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

And your repository is ready for use. You can now install Kibana with one of the following command:

sudo yum install kibana

Running Kibana with systemd

To configure Kibana to start automatically when the system boots up, run the following commands:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable kibana.service

Kibana can be started and stopped as follows:

sudo systemctl stop kibana.service
sudo systemctl start kibana.service

These commands provide no feedback as to whether Kibana was started successfully or not. Instead, this information will be written in the log files located in /var/log/kibana/. Kibana loads its configuration from the /etc/kibana/kibana.yml file by default. Examples:

• elasticsearch.url: Default: http://localhost:9200 The URL of the Elasticsearch instance to use for all your queries.
• server.port: Server port for the Kibana web UI - default 5601
• server.host: Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values. The default is localhost, which usually means remote machines will not be able to connect. To allow connections from remote users, set this parameter to a non-loopback address.
• console.enabled: Default: true Set to false to disable Console.
• elasticsearch.username: s. below
• elasticsearch.password: If your Elasticsearch is protected with basic authentication, these settings provide the username and password that the Kibana server uses to perform maintenance on the Kibana index at startup. Your Kibana users still need to authenticate with Elasticsearch, which is proxied through the Kibana server. (see X-Pack below)
• server.ssl.enabled: Default: 'false' Enables SSL for outgoing requests from the Kibana server to the browser. When set to true, server.ssl.certificate and server.ssl.key are required
• server.ssl.certificate: s. below
• server.ssl.key: Paths to the PEM-format SSL certificate and SSL key files, respectively.
• server.ssl.certificateAuthorities: List of paths to PEM encoded certificate files that should be trusted.
• server.ssl.cipherSuites: Default: ECDHE-RSA-AES128-GCM-SHA256, ECDHE-ECDSA-AES128-GCM-SHA256, ECDHE-RSA-AES256-GCM-SHA384, ECDHE-ECDSA-AES256-GCM-SHA384, DHE-RSA-AES128-GCM-SHA256, ECDHE-RSA-AES128-SHA256, DHE-RSA-AES128-SHA256, ECDHE-RSA-AES256-SHA384, DHE-RSA-AES256-SHA384, ECDHE-RSA-AES256-SHA256, DHE-RSA-AES256-SHA256, HIGH,!aNULL, !eNULL, !EXPORT, !DES, !RC4, !MD5, !PSK, !SRP, !CAMELLIA. Details on the format, and the valid options, are available via the OpenSSL cipher list format documentation
• server.ssl.keyPassphrase: The passphrase that will be used to decrypt the private key. This value is optional as the key may not be encrypted.
• server.ssl.redirectHttpFromPort: Kibana will bind to this port and redirect all http requests to https over the port configured as server.port.
• server.ssl.supportedProtocols: Default: TLSv1, TLSv1.1, TLSv1.2 Supported protocols with versions. Valid protocols: TLSv1, TLSv1.1, TLSv1.2
• status.allowAnonymous: Default: false If authentication is enabled, setting this to true allows unauthenticated users to access the Kibana server status API and status page.

Type	Description	Default Location	Setting
home	Kibana home directory or $KIBANA_HOME	/usr/share/kibana
bin	Binary scripts including kibana to start the Kibana server and kibana-plugin to install plugins	/usr/share/kibana/bin
config	Configuration files including kibana.yml	/etc/kibana
data	The location of the data files written to disk by Kibana and its plugins	/var/lib/kibana	path.data
optimize	Transpiled source code. Certain administrative actions (e.g. plugin install) result in the source code being retranspiled on the fly.	/usr/share/kibana/optimize
plugins	Plugin files location. Each plugin will be contained in a subdirectory.	/usr/share/kibana/plugins

---

Useful Links:

• Elasticsearch Cookbook
• Ansible Elasticsearch
• Deploying Elasticsearch
• Chef.io
• Maintain Security
• Secure Elasticsearch and Kibana