Elasticsearch custom container on Azure App Services

In this blog, we will cover how to deploy Elasticsearch container on Azure App Service

we are using the container image elasticsearch:7.17.25 available on Docker Hub elasticsearch - Official Image | Docker Hub

While deploying the container image on Azure App Service, the Docker container encounters the following error:

2024-11-14T12:37:41.643855847Z {"type": "server", "timestamp": "2024-11-14T12:37:41,643Z", "level": "INFO", "component": "o.e.x.m.Monitoring", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "creating template [.monitoring-es] with version [7]" }
2024-11-14T12:37:41.645055848Z {"type": "server", "timestamp": "2024-11-14T12:37:41,644Z", "level": "INFO", "component": "o.e.x.m.Monitoring", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "creating template [.monitoring-kibana] with version [7]" }
2024-11-14T12:37:41.646677131Z {"type": "server", "timestamp": "2024-11-14T12:37:41,646Z", "level": "INFO", "component": "o.e.x.m.Monitoring", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "creating template [.monitoring-logstash] with version [7]" }
2024-11-14T12:37:41.650269656Z {"type": "server", "timestamp": "2024-11-14T12:37:41,650Z", "level": "INFO", "component": "o.e.x.m.Monitoring", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "creating template [.monitoring-beats] with version [7]" }
2024-11-14T12:37:41.733555119Z {"type": "server", "timestamp": "2024-11-14T12:37:41,733Z", "level": "INFO", "component": "o.e.b.BootstrapChecks", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "bound or publishing to a non-loopback address, enforcing bootstrap checks" }
2024-11-14T12:37:41.738339565Z
2024-11-14T12:37:41.738374780Z ERROR: [2] bootstrap checks failed. You must address the points described in the following [2] lines before starting Elasticsearch.
2024-11-14T12:37:41.738387734Z bootstrap check failure [1] of [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
2024-11-14T12:37:41.738395509Z bootstrap check failure [2] of [2]: the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
2024-11-14T12:37:41.738446001Z ERROR: Elasticsearch did not exit normally - check the logs at /usr/share/elasticsearch/logs/docker-cluster.log
2024-11-14T12:37:41.740094676Z {"type": "server", "timestamp": "2024-11-14T12:37:41,739Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "stopping ..." }
2024-11-14T12:37:41.748283513Z {"type": "server", "timestamp": "2024-11-14T12:37:41,748Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "stopped" }
2024-11-14T12:37:41.748398709Z {"type": "server", "timestamp": "2024-11-14T12:37:41,748Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "closing ..." }
2024-11-14T12:37:41.755743691Z {"type": "server", "timestamp": "2024-11-14T12:37:41,755Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "closed" }
2024-11-14T12:37:41.757215226Z {"type": "server", "timestamp": "2024-11-14T12:37:41,756Z", "level": "INFO", "component": "o.e.x.m.p.NativeController", "cluster.name": "docker-cluster", "node.name": "e6413475d15a", "message": "Native controller process has stopped - no new native processes can be started" }

Since Azure App Services is a Platform as a Service (PaaS), you do not have the privileges to modify the vm.max_map_count.

We can mitigate the issue by adding the app setting discovery.type=single-node, please refer Bootstrap Checks | Elasticsearch Guide [8.16] | Elastic for more details

But you would notice that the container still failing with the same error as the app setting name was modified. You can check the app settings from the kudu "newui" portal

Navigate to https://<AppServiceName>.scm.azurewebsites.net/newui --> Environment tab

By default, Azure App Service on Linux or WebApp for Containers replaces any periods in app settings with a single underscore (_)

Please refer Configure apps - Azure App Service | Microsoft Learn for more details.

To address the issue, we will configure discovery.type in the elasticsearch.yml file during the container image build process.

Below are the steps to build, run, and push the image to a repository.

1. Please create a folder named "test" and then add Dockerfile and elasticsearch.yml files with the following contents

Dockerfile

Use the Elasticsearch base image
FROM elasticsearch:7.17.25
# Set the environment variable path to override the default configuration
COPY elasticsearch.yml /usr/share/elasticsearch/config

elasticsearch.yml

cluster.name: "docker-cluster"

network.host: 0.0.0.0

discovery.type: single-node

 Test folder structure 

2. Build and run the docker image
Docker Build Command: docker build -t custom-elasticsearch:7.17.25

Docker Run Command: docker run -d -p 1234:9200 custom-elasticsearch:7.17.25

3. Browse the application via https://localhost:1234

4. Now, proceed to push the running Docker image to the Azure Container Registry (ACR) or any other repository of your preference.

az acr login -n <ACRname>

docker tag custom-elasticsearchlatest:7.17.25 acrname.azurecr.io/custom-elasticsearch:1

docker push acrname.azurecr.io/custom-elasticsearch:1

5. Given that the Elasticsearch Docker image is resource-intensive, please ensure you use a high-memory configuration App Service Plan (ASP) to provide adequate memory.

6. Please update the app service with the latest Docker image and tag and ensure to add the application setting WEBSITES_PORT=9200, as the container is listening on port 9200.

7. The container has started successfully, and the app service is functioning without any issues.

Happy Learning 🙂