Blue/Green Deployment on Kubernetes using Helm Charts and Istio Gateway

Why we need Blue/Green deployment strategy?

• Blue/Green strategy is used to minimize downtime for production services.
• There are two identical production environments as known as Blue/Green deployments to represent Prod/Stag environments, and only one of them is serving the production traffic.
• When a new release need to be deployed, we will check the which deployment is serving the production traffic.

Blue Deployment is serving the production traffic.

• For example, the Blue deployment is the Prod environment, and we will deploy new release to the Green deployment for the Stag environment. When all the testing is done, the live traffic is moved to Stage which becomes the Prod environment.

Switch to Green deployment to serve the production traffic.

• If we need to make a fast rollback by just changing the route weight for the Blue/Green services if new issues are found with live traffic.

In following let’s show that how we can achieve Blue/Green deployments on Kubernetes using Helm chart and Istio gateway. Here we are defined some values in values.yaml which are be used by Helm Chart. Let’s talk about how to work based on those values.

nameOverride: bg-deploy
green:
  appVersion: 1.0
  enabled: false
  timestamp: "2021-05-01 00:00:00"
blue:
  appVersion: 1.0
  enabled: false
  timestamp: "2021-05-01 00:00:00"
productionSlot: blue
replicaCount: 1

nameOverride: The default name is constructed from the release name and the chart name, where nameOverride overrides the chart name.

green / blue: Here we call the two environments slots, i.e. blueslot and greenslot. For each slot has its own values are described below:

appVersion: The application version for this release.

enabled: This slot is be enabled.

timestamp: The time for this slot is be deployed.

productionSlot:The slot is serving the production traffic. We use the value to determine which slot need to set the weight 100 and the other slot set to weight 0.

replicaCount:The amount of pods in each of green / blue deployment.

Define the blue deployment and service file in the Helm Chart template folder.

{{ if .Values.blue.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ template "blue-green.fullname" . }}-blue
  labels:
    release: {{ .Release.Name }}
    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
    app: {{ template "blue-green.name" . }}
    version: {{ .Values.blue.appVersion | quote }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ template "blue-green.name" . }}
      version: {{ .Values.blue.appVersion | quote }}
      release: {{ .Release.Name }}
      slot: blue
  template:
    metadata:
      labels:
        app: {{ template "blue-green.name" . }}
        version: {{ .Values.blue.appVersion | quote }}
        release: {{ .Release.Name }}
        slot: blue
    spec:
      containers:
        - name: {{ template "blue-green.name" . }}-blue
          env:
            - name: APP_COLOR
              value: blue
            - name: CURRENT_TIMESTAMP
              value: {{ .Values.blue.timestamp }}
          image: kodekloud/webapp-color:latest
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          # This (and the volumes section below) mount the config map as a volume.
          volumeMounts:
            - mountPath: /usr/share/nginx/html
              name: wwwdata-volume
      volumes:
        - name: wwwdata-volume
          emptyDir: {}
{{ end }}

Define the green deployment and service file in the Helm Chart template folder.

{{ if .Values.green.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ template "blue-green.fullname" . }}-green
  labels:
    release: {{ .Release.Name }}
    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
    app: {{ template "blue-green.name" . }}
    version: {{ .Values.green.appVersion | quote }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ template "blue-green.name" . }}
      version: {{ .Values.green.appVersion | quote  }}
      release: {{ .Release.Name }}
      slot: green
  template:
    metadata:
      labels:
        app: {{ template "blue-green.name" . }}
        version: {{ .Values.green.appVersion | quote }}
        release: {{ .Release.Name }}
        slot: green
    spec:
      containers:
        - name: {{ template "blue-green.name" . }}-green
          env:
            - name: APP_COLOR
              value: green
            - name: CURRENT_TIMESTAMP
              value: {{ .Values.green.timestamp }}
          image: kodekloud/webapp-color:latest
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 8080
              protocol: TCP
          # This (and the volumes section below) mount the config map as a volume.
          volumeMounts:
            - mountPath: /usr/share/nginx/html
              name: wwwdata-volume
      volumes:
        - name: wwwdata-volume
          emptyDir: {}
{{ end }}

Define the gateway-istio.yaml file in the Helm Chart template folder.

You can replace your hostname with the sample domain: deploy-test.example.org

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: {{ include "blue-green.fullname" . }}-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http-tracing
      protocol: HTTP
    hosts:
    - deploy-test.example.org
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: {{ include "blue-green.fullname" . }}-vs
spec:
  hosts:
  - deploy-test.example.org
  gateways:
  - {{ include "blue-green.fullname" . }}-gateway
  http:
  - route:
    - destination:
        host: {{ include "blue-green.name" . }}-service-blue
        port:
          number: 8080
    {{- if eq .Values.productionSlot "blue" }}
      weight: 100
    {{- else }}
      weight: 0
    {{- end }}
    - destination:
        host: {{ include "blue-green.name" . }}-service-green
        port:
          number: 8080
    {{- if eq .Values.productionSlot "blue" }}
      weight: 0
    {{- else }}
      weight: 100
    {{- end }}
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: {{ include "blue-green.fullname" . }}-dr
spec:
  host: {{ include "blue-green.fullname" . }}
  trafficPolicy:
    tls:
      mode: DISABLE

Demostration

You can download the source code from my Github here and some prerequisites need to be installed on your Kubernetes cluster first.

Let’s show a demonstration with the following steps:

• Change directory to the root folder of blue-green helm chart

$ cd blue-green

• The first time deploy the blue and green deployment with the version “1.0” for both slots by the Helm install command and than switch traffic to blue

$ helm install deploy-test . --namespace=my-app --set blue.enabled=true --set blue.timestamp="$(date '+%Y-%m-%d %H:%M:%S')" --set blue.appVersion="1.0" --set green.timestamp="$(date '+%Y-%m-%d %H:%M:%S')" --set green.appVersion="1.0" --set productionSlot=blue --debug

• Deploy the green deployment with the new version “1.1” by the Helm upgrade command

$ ./deployBlueOrGreen.sh green 1.1

• Checkout the rolling update status is ready

$ kubectl rollout status deployments/bg-deploy-green -n my-app
deployment "deploy-test-my-app-green" successfully rolled out

• Switch the traffic to green

$ ./switchTraffic.sh green

• Repeat the steps for upgrading the blue deployment with new version “1.2” again

$ ./deployBlueOrGreen.sh blue 1.2
$ kubectl rollout status deployments/bg-deploy-blue -n my-app
$ ./moveTraffic.sh blue

• You can monitor live traffic with the Kiali graph. The Kiali graph provides a powerful way to visualize the topology of your service mesh. The installation guide for Kiali is here.

Reference

• https://medium.com/harness-engineering/blue-green-deployments-using-helm-charts-93ec479c0282