Open Lightweight Service Mesh for K8s

[monojit18]

What is Service Mesh?

Service Mesh provides managing capabilities for the micro-services hosted in the Kubernetes cluster uniformly and decouples them from the Applications (Microservices) layer.

This makes it a great tool for Infrastructure architects as well as Developer architects easily Manage, Observe and Secure the cluster.

Features

• Observability - Monitor applications hosted inside the cluster

• Traffic Splitting - Split Ingress Traffic to different versions of an API. Helps perform - Blue/Green Deployment, A/B Testing, Fault Injection

• Distributed Tracing - Trace/Compare Service to Service calls with visualisation

• Circuit Breaking - Limit impact of failures due to uncertainty in Network and make services resilient

Considerations

• Complexity - More complex for simple requirements

• Resource Overhead - Additional components needs additional resources - CPU, Memory

• Additional Latency - Proxies, Policy checks adds latency; so might be a challenge for highly latency sensitive applications

Open Service Mesh

Open Service Mesh is a Lightweight, Extensible Service Mesh tool designed to manage and secure APIs inside K8s cluster by introducing simplicity and reducing complexity.

It is based on envoy Proxy and injects this as a sidecar container into every Observable application which in-turn performs traffic management, routing policies, capturing metrics etc.

OSM Features

And what is meant by Observable - Users can choose which of the applications (namespaces) should be under the OSM scanner and OSM would monitor those leaving others untouched!

Here is Quick Reference on OSM and Demo to understand how it works!

This article would show how this demo can be setup on an AKS cluster and can be used to observer, manage applications hosted inside the AKS cluster.

Please note that AKS now has an add-on for OSM.

Let us now build the demo step-by-step on an existing cluster.

Please refer to the Source Code for OSM which we would be referring in this article.

Define CLI Variables

tenantId=""
subscriptionId=""
location="eastus"
clusterName="aks-train-mesh-cluster"
version=""
aksResourceGroup="aks-train-rg"
acrName="srvmeshacr"
osmFolderPath="<osm-folder-path-in-repo>"

Please fill up the values appropriately.

Connect to the AKS cluster

az aks get-credentials -g $aksResourceGroup --name $clusterName --admin --overwrite

Install Open Service Mesh (OSM)

osm install \
    --set=OpenServiceMesh.enablePermissiveTrafficPolicy=false \
    --set=OpenServiceMesh.deployPrometheus=true \
    --set=OpenServiceMesh.deployGrafana=true \
    --set=OpenServiceMesh.deployJaeger=true

Create Namespaces

kubectl create namespace bookstore
kubectl create namespace bookbuyer
kubectl create namespace bookthief
kubectl create namespace bookwarehouse

 

Inject OSM into the Namespaces

osm namespace add bookstore
osm namespace add bookbuyer
osm namespace add bookthief
osm namespace add bookwarehouse

Enable Metrics for OSM Namespaces

osm metrics enable --namespace bookstore
osm metrics enable --namespace bookbuyer
osm metrics enable --namespace bookthief
osm metrics enable --namespace bookwarehouse

Import images from Docker Hub

#bookbuyer
az acr import -n $acrName --source docker.io/openservicemesh/bookbuyer:v0.11.1 -t bookbuyer:v0.11.1
​
#bookstore
az acr import -n $acrName --source docker.io/openservicemesh/bookstore:v0.11.1 -t bookstore:v0.11.1
​
#bookthief
az acr import -n $acrName --source docker.io/openservicemesh/bookthief:v0.11.1 -t bookthief:v0.11.1
​
#bookwarehouse
az acr import -n $acrName --source docker.io/openservicemesh/bookwarehouse:v0.11.1 -t bookwarehouse:v0.11.1

Deploy Applications

#Deploy bookbuyer
kubectl apply -f $osmfolderPath/yamls/bookbuyer.yaml
​
#Deploy bookstore
kubectl apply -f $osmfolderPath/yamls/bookstore.yaml
​
#Deploy bookthief
kubectl apply -f $osmfolderPath/yamls/bookthief.yaml
​
#Deploy bookwarehouse
kubectl apply -f $osmfolderPath/yamls/bookwarehouse.yaml

Check Deployments

kubectl get deployments -n bookbuyer
kubectl get deployments -n bookthief
kubectl get deployments -n bookstore
kubectl get deployments -n bookwarehouse
​
kubectl get pods -n bookbuyer
kubectl get pods -n bookthief
kubectl get pods -n bookstore
kubectl get pods -n bookwarehouse
​
kubectl get services -n bookstore
kubectl get services -n bookwarehouse
​
kubectl get endpoints -n bookstore
kubectl get endpoints -n bookwarehouse

Check Mesh configuration

kubectl get meshconfig osm-mesh-config -n osm-system -o yaml

How to expose services to the outside world?

This is primarily done using an Ingress controller.

Below steps would integrate OSM with Nginx Ingress controller so that all traffic from Ingress passes thru OSM and be monitored through the mesh

Install Nginx Ingress

#Create namespace for Nginx Ingress
kubectl create ns osm-nginx-ingess-ns
​
#Install Nginx Ingress controller
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
​
helm install osm-nginx-ingess ingress-nginx/ingress-nginx --namespace osm-nginx-ingess-ns \
--set controller.nodeSelector.agentpool=agentpool \
--set controller.defaultBackend.nodeSelector.agentpool=agentpool

Ingress to route to BookStore Service

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: osm-ingress
  namespace: bookstore
  annotations:
    kubernetes.io/ingress.class: nginx    
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  # tls:
  # - hosts:
  #   - store.<dns-name>
  #   secretName: osm-tls-secret
  rules:
  - host: store.<dns-name>
    http:
      paths:
      - path: /store
        pathType: Prefix
        backend:
          service:
            name: bookstore
            port:
              number: 14001
​
---
kind: IngressBackend
apiVersion: policy.openservicemesh.io/v1alpha1
metadata:
  name: bookstore-backend
  namespace: bookstore
spec:
  backends:
  - name: bookstore
    port:
      number: 14001
      protocol: http
  sources:
  - kind: Service
    namespace: osm-nginx-ingess-ns
    name: osm-nginx-ingess-ingress-nginx-controller

 

Access underlying services

• BookThief

  #Run BookThief as background service
  ./$osmFolderPath/osm/scripts/port-forward-bookthief-ui.sh &
  ​
  #Open BookThief page in browser
  http://localhost:8083/

• BookBuyer

  #Run BookBuyer as background service
  ./$osmFolderPath/osm/scripts/port-forward-bookbuyer-ui.sh &
  ​
  #Open BookBuyer page in browser
  http://localhost:8080/

• Check that services are Not working

Deploy Traffic Target

• Allows traffic between services within the Mesh

kind: TrafficTarget
apiVersion: access.smi-spec.io/v1alpha3
metadata:
  name: bookstore
  namespace: bookstore
spec:
  destination:
    kind: ServiceAccount
    name: bookstore
    namespace: bookstore
  rules:
  - kind: HTTPRouteGroup
    name: bookstore-service-routes
    matches:
    - buy-a-book
    - books-bought
  sources:
  - kind: ServiceAccount
    name: bookbuyer
    namespace: bookbuyer
  - kind: ServiceAccount
    name: bookthief
    namespace: bookthief
---
apiVersion: specs.smi-spec.io/v1alpha4
kind: HTTPRouteGroup
metadata:
  name: bookstore-service-routes
  namespace: bookstore
spec:
  matches:
  - name: books-bought
    pathRegex: /books-bought
    methods:
    - GET
    headers:
    - "user-agent": ".*-http-client/*.*"
    - "client-app": "bookbuyer"
  - name: buy-a-book
    pathRegex: ".*a-book.*new"
    methods:
    - GET

kubectl apply -f $osmFolderPath/yamls/traffic-access.yaml

• Check Traffic starts flowing through

Deploy BookStore-V2

kubectl apply -f $osmFolderPath/yamls/bookstore-v2.yaml

Split Traffic

• Split traffic between Two versions of BookStore service - 50/50

  kubectl apply -f $osmFolderPath/yamls/traffic-split-50-50.yaml

  apiVersion: split.smi-spec.io/v1alpha2
  kind: TrafficSplit
  metadata:
    name: bookstore-split
    namespace: bookstore
  spec:
    service: bookstore.bookstore # <root-service>.<namespace>
    backends:
    - service: bookstore
      weight: 50
    - service: bookstore-v2
      weight: 50

• Split traffic between Two versions of BookStore service - 0/100

  kubectl apply -f $osmFolderPath/yamls/traffic-split-v2.yaml

  apiVersion: split.smi-spec.io/v1alpha2
  kind: TrafficSplit
  metadata:
    name: bookstore-split
    namespace: bookstore
  spec:
    service: bookstore.bookstore # <root-service>.<namespace>
    backends:
    - service: bookstore
      weight: 0
    - service: bookstore-v2
      weight: 100
  ​

 

OSM Metrics

osm dashboard&

OSM Tracing

kubectl patch meshconfig osm-mesh-config -n osm-system -p '{"spec":{"observability":{"tracing":{"enable":true,"address": "jaeger.osm-system.svc.cluster.local","port":9411,"endpoint":"/api/v2/spans"}}}}'  --type=merge

OSM Egress

• Egress is pods to the outside world is blocked by default

• Deploy curl application

  kubectl create ns curl
  ​
  kubectl apply -f $osmFolderPath/yamls/curl.yaml
  ​
  kubectl get pods -n curl

• Go Inside the curl pod

  kubectl exec -it <curl-pod-name> -n curl -- sh
  ​
  curl http://httpbin.org/get

• Check Egress from curl pod fails

• Now Enable Egress for Pods

kubectl patch meshconfig osm-mesh-config -n osm-system -p '{"spec":{"featureFlags":{"enableEgressPolicy":true}}}'  --type=merge

• Go Inside the curl pod again

kubectl exec -it <curl-pod-name> -n curl -- sh
​
curl http://httpbin.org/get

• Check Egress from curl pod working as expected

Cleanup

• List all Namespaces under OSM scanner

  osm ns list --mesh-name=osm

• Remove Namespaces from OSM scanner

  osm namespace remove bookbuyer --mesh-name=osm
  osm namespace remove bookstore --mesh-name=osm
  osm namespace remove bookthief --mesh-name=osm
  osm namespace remove bookwarehouse --mesh-name=osm

• Restart the deployment as Envoy sidecars are deleted

  kubectl rollout restart deployment bookbuyer -n bookbuyer
  kubectl rollout restart deployment bookstore -n bookstore
  kubectl rollout restart deployment bookthief -n bookthief
  kubectl rollout restart deployment bookwarehouse -n bookwarehouse

• UnInstall OSM from AKS cluster

  osm uninstall --mesh-name=osm