How to install Elasticsearch on Ubuntu 20.04 LTS

BitLaunch -

Sign up for BitLaunch and install Elasticsearch on an Ubuntu 20.04 KVM VPS.

Introduction

Elasticsearch is a distributed search and analytics engine based on JSON data. Its powerful search mechanisms made it popular for many applications. Though it's developed in Java, there are official Elasticsearch Python, Ruby, .NET, and other clients, making it a versatile tool.

This Elasticsearch tutorial will walk you through the install of Elasticsearch on Ubuntu 20.04 LTS, with an additional section on how to secure it.

Installing Elastic Search

You will install Elasticsearch on Ubuntu using it's official package repo, but first you need to import it's GPG key to verify the package.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

Once the key is imported following output will be displayed:

OK

As a requirement you need to install the apt-transport-https package.

$ sudo apt-get install -y apt-transport-https

An output similar to following will be displayed:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
...

Next, you need to add the repo to your apt sources list.

$ echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

The output should be as follows:

deb https://artifacts.elastic.co/packages/7.x/apt stable main

Update the package index, using the apt update command.

$ sudo apt update

The following output will be displayed.

Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Get:4 https://artifacts.elastic.co/packages/7.x/apt stable InRelease [10.4 kB]
Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease                  
Get:6 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 Packages [38.2 kB]
Fetched 48.6 kB in 1s (84.9 kB/s)  
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.

Now you can install elasticsearch package.

$ sudo apt install -y elasticsearch

Enable Elasticsearch to start on boot with the following command.

$ sudo systemctl enable elasticsearch

Following output will be displayed.

Synchronizing state of elasticsearch.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable elasticsearch
Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /lib/systemd/system/elasticsearch.service.

You can now start Elasticsearch.

$ sudo systemctl start  elasticsearch

Check the status of the Elasticsearch.

$ sudo systemctl status  elasticsearch

systemctl output will be as following.

● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-07-18 06:12:29 UTC; 25s ago
       Docs: https://www.elastic.co
   Main PID: 2191 (java)
      Tasks: 69 (limit: 614)
     Memory: 1.2G
     CGroup: /system.slice/elasticsearch.service
             ├─2191 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m >
             └─2384 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Jul 18 06:12:05 5f0c1d79a6fdb60001eb24f0 systemd[1]: Starting Elasticsearch...
Jul 18 06:12:29 5f0c1d79a6fdb60001eb24f0 systemd[1]: Started Elasticsearch.

As Elasticsearch has started you can query it usign curl.

$ curl http://localhost:9200/

A similar to following output will be displayed.

{
  "name" : "5f0c1d79a6fdb60001eb24f0",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "50RGMvEFRV-tP-99bu9BKg",
  "version" : {
    "number" : "7.8.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
    "build_date" : "2020-06-14T19:35:50.234439Z",
    "build_snapshot" : false,
    "lucene_version" : "8.5.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Securing ElasticSearch

The first step to securing Elasticsearch is enabling password protection. Elasticsearch has a tool to setup passwords, using the auto option, you can let Elasticsearch generate passwords automatically.

$ sudo /usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto

An output similar to the following will be displayed, with randomly generated passwords are shown as below:

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
The passwords will be randomly generated and printed to the console.
Please confirm that you would like to continue [y/N]y


Changed password for user apm_system
PASSWORD apm_system = 6F9Bj7rfEUABJjPcqpHv

Changed password for user kibana_system
PASSWORD kibana_system = UEKh4kGlNw4hVCRBrVWs

Changed password for user kibana
PASSWORD kibana = UEKh4kGlNw4hVCRBrVWs

Changed password for user logstash_system
PASSWORD logstash_system = bkIqDRX0QDEcJuqDZ2C6

Changed password for user beats_system
PASSWORD beats_system = X4I6laXnZyEE2QJaRbeL

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = lRrj8zchlE6J8SAMgF9H

Changed password for user elastic
PASSWORD elastic = oEskdwuPbCSzCLbgub57

Next, you will generate the certificate and the key using the openssl command

$ sudo openssl req -x509 -nodes -newkey rsa:4096 -keyout /etc/elasticsearch/config/es-key.pem -out /etc/elasticsearch/config/es-cert.pem -days 365

Change the file permissions so that the elasticsearch user can also read the key file.

$ sudo chmod 640 /etc/elasticsearch/config/es-key.pem

Open Elasticsearch's configuration file with your favorite editor, this guide uses nano.

$ sudo nano /etc/elasticsearch/elasticsearch.yml

Paste the following at the end of the file.

xpack.security.enabled: true

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /etc/elasticsearch/config/es-key.pem
xpack.security.http.ssl.certificate: /etc/elasticsearch/config/es-cert.pem

This configuration enables security options so that password protection is enabled. Next you define the SSL certificate and key file.

Restart Elasticsearch with systemctl so that the new configuration takes effect.

$ sudo systemctl restart elasticsearch.service

Now you can test via the curl command without a password to make sure password protection is enabled. Note that as a self-signed certificate is used, the -k option is needed to accept the certificate.

curl -k https://localhost:9200/

Following output tells us authentication is required.

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}

Retest with the correct username and password.

$ curl -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/

The following output shows our username and password is correct and we are accessing over HTTPS.

{
  "name" : "5f0c1d79a6fdb60001eb24f0",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "50RGMvEFRV-tP-99bu9BKg",
  "version" : {
    "number" : "7.8.0",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
    "build_date" : "2020-06-14T19:35:50.234439Z",
    "build_snapshot" : false,
    "lucene_version" : "8.5.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Testing ElasticSearch

To create a document you will use curl and POST.

$ curl -X POST -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/testindex/_doc?pretty -H 'Content-Type: application/json' -d'{ "message" : "trying out Elasticsearch" }'

Output is as follows:

{
  "_index" : "testindex",
  "_type" : "_doc",
  "_id" : "zg2iaHMB3GD9n7JC7clS",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 3,
  "_primary_term" : 1
}

To retreive a document you can use HTTP Get using document id.

$ curl -X GET  -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/testindex/_doc/zg2iaHMB3GD9n7JC7clS?pretty

{
  "_index" : "testindex",
  "_type" : "_doc",
  "_id" : "zg2iaHMB3GD9n7JC7clS",
  "_version" : 1,
  "_seq_no" : 3,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "message" : "trying out Elasticsearch"
  }
}

To delete a document, use HTTP delete and the document id.

$ curl -X DELETE  -k --user elastic:oEskdwuPbCSzCLbgub57 https://localhost:9200/testindex/_doc/zg2iaHMB3GD9n7JC7clS?pretty

{
  "_index" : "testindex",
  "_type" : "_doc",
  "_id" : "zg2iaHMB3GD9n7JC7clS",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 4,
  "_primary_term" : 1
}

Conclusion

You now know how to install Elasticsearch and secure it with a password and SSL. Now you can search to your heart's content, or customize the tool to suit whatever other purpose you plan to use it for.

References

ElasticSearch Documentation