Sign up for BitLaunch and install Elasticsearch on an Ubuntu 20.04 KVM VPS.
Introduction
Elasticsearch is a distributed search and analytics engine based on JSON data. Its powerful search mechanisms made it popular for many applications. Though it's developed in Java, there are official Elasticsearch Python, Ruby, .NET, and other clients, making it a versatile tool.
This Elasticsearch tutorial will walk you through the install of Elasticsearch on Ubuntu 20.04 LTS, with an additional section on how to secure it.
Installing Elastic Search
You will install Elasticsearch on Ubuntu using it's official package repo, but first you need to import it's GPG key to verify the package.
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
Once the key is imported following output will be displayed:
OK
As a requirement you need to install the apt-transport-https package.
$ sudo apt-get install -y apt-transport-https
An output similar to following will be displayed:
Reading package lists... Done
Building dependency tree
Reading state information... Done
...
Next, you need to add the repo to your apt sources list.
$ echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
The output should be as follows:
deb https://artifacts.elastic.co/packages/7.x/apt stable main
Update the package index, using the apt update command.
$ sudo apt update
The following output will be displayed.
Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Get:4 https://artifacts.elastic.co/packages/7.x/apt stable InRelease [10.4 kB]
Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease
Get:6 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 Packages [38.2 kB]
Fetched 48.6 kB in 1s (84.9 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
Now you can install elasticsearch package.
$ sudo apt install -y elasticsearch
Enable Elasticsearch to start on boot with the following command.
$ sudo systemctl enable elasticsearch
Following output will be displayed.
Synchronizing state of elasticsearch.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable elasticsearch
Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /lib/systemd/system/elasticsearch.service.
You can now start Elasticsearch.
$ sudo systemctl start elasticsearch
Check the status of the Elasticsearch.
$ sudo systemctl status elasticsearch
systemctl output will be as following.
● elasticsearch.service - Elasticsearch
Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2020-07-18 06:12:29 UTC; 25s ago
Docs: https://www.elastic.co
Main PID: 2191 (java)
Tasks: 69 (limit: 614)
Memory: 1.2G
CGroup: /system.slice/elasticsearch.service
├─2191 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m >
└─2384 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
Jul 18 06:12:05 5f0c1d79a6fdb60001eb24f0 systemd[1]: Starting Elasticsearch...
Jul 18 06:12:29 5f0c1d79a6fdb60001eb24f0 systemd[1]: Started Elasticsearch.
As Elasticsearch has started you can query it usign curl.
$ curl http://localhost:9200/
A similar to following output will be displayed.
{
"name" : "5f0c1d79a6fdb60001eb24f0",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "50RGMvEFRV-tP-99bu9BKg",
"version" : {
"number" : "7.8.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
"build_date" : "2020-06-14T19:35:50.234439Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Securing ElasticSearch
The first step to securing Elasticsearch is enabling password protection. Elasticsearch has a tool to setup passwords, using the auto option, you can let Elasticsearch generate passwords automatically.
$ sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto
An output similar to the following will be displayed, with randomly generated passwords are shown as below:
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y
Changed password for user apm_system
PASSWORD apm_system = 6F9Bj7rfEUABJjPcqpHv
Changed password for user kibana_system
PASSWORD kibana_system = UEKh4kGlNw4hVCRBrVWs
Changed password for user kibana
PASSWORD kibana = UEKh4kGlNw4hVCRBrVWs
Changed password for user logstash_system
PASSWORD logstash_system = bkIqDRX0QDEcJuqDZ2C6
Changed password for user beats_system
PASSWORD beats_system = X4I6laXnZyEE2QJaRbeL
Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = lRrj8zchlE6J8SAMgF9H
Changed password for user elastic
PASSWORD elastic = oEskdwuPbCSzCLbgub57
Next, you will generate the certificate and the key using the openssl command
$ sudo openssl req -x509 -nodes -newkey rsa:4096 -keyout /etc/elasticsearch/config/es-key.pem -out /etc/elasticsearch/config/es-cert.pem -days 365
Change the file permissions so that the elasticsearch user can also read the key file.
$ sudo chmod 640 /etc/elasticsearch/config/es-key.pem
Open Elasticsearch's configuration file with your favorite editor, this guide uses nano.
$ sudo nano /etc/elasticsearch/elasticsearch.yml
Paste the following at the end of the file.
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/config/es-key.pem
xpack.security.http.ssl.certificate: /etc/elasticsearch/config/es-cert.pem
This configuration enables security options so that password protection is enabled. Next you define the SSL certificate and key file.
Restart Elasticsearch with systemctl so that the new configuration takes effect.
$ sudo systemctl restart elasticsearch.service
Now you can test via the curl command without a password to make sure password protection is enabled. Note that as a self-signed certificate is used, the -k option is needed to accept the certificate.
curl -k https://localhost:9200/
Following output tells us authentication is required.
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}
Retest with the correct username and password.
$ curl -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/
The following output shows our username and password is correct and we are accessing over HTTPS.
{
"name" : "5f0c1d79a6fdb60001eb24f0",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "50RGMvEFRV-tP-99bu9BKg",
"version" : {
"number" : "7.8.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
"build_date" : "2020-06-14T19:35:50.234439Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Testing ElasticSearch
To create a document you will use curl and POST.
$ curl -X POST -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/testindex/_doc?pretty -H 'Content-Type: application/json' -d'{ "message" : "trying out Elasticsearch" }'
Output is as follows:
{
"_index" : "testindex",
"_type" : "_doc",
"_id" : "zg2iaHMB3GD9n7JC7clS",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 3,
"_primary_term" : 1
}
To retreive a document you can use HTTP Get using document id.
$ curl -X GET -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/testindex/_doc/zg2iaHMB3GD9n7JC7clS?pretty
{
"_index" : "testindex",
"_type" : "_doc",
"_id" : "zg2iaHMB3GD9n7JC7clS",
"_version" : 1,
"_seq_no" : 3,
"_primary_term" : 1,
"found" : true,
"_source" : {
"message" : "trying out Elasticsearch"
}
}
To delete a document, use HTTP delete and the document id.
$ curl -X DELETE -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/testindex/_doc/zg2iaHMB3GD9n7JC7clS?pretty
{
"_index" : "testindex",
"_type" : "_doc",
"_id" : "zg2iaHMB3GD9n7JC7clS",
"_version" : 2,
"result" : "deleted",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 4,
"_primary_term" : 1
}
Conclusion
You now know how to install Elasticsearch and secure it with a password and SSL. Now you can search to your heart's content, or customize the tool to suit whatever other purpose you plan to use it for.