The Distributed SQL Blog

Thoughts on distributed databases, open source and cloud native

Service Mesh Era: Building Modern Apps with YugabyteDB and Istio

Microservices architectures are becoming the de facto way developers are thinking about how their applications are constructed. But security remains a top concern for many organizations. Given the general trends of the proliferation of threats within the production network and the increased points of privileged access, it is increasingly necessary to adopt a zero-trust network security approach for microservices architectures.

One of the most common security approaches is to set up mTLS. While this is an important security tool, it’s often difficult and time consuming to manage. To start, you have to create, distribute, and rotate keys and certificates to a large number of services. You then need to ensure you are properly implementing mTLS on all of your clients and servers. One of the compelling features of Istio is the ability to uniformly administer mTLS for all of your services without sacrificing developer productivity. While it’s true YugabyteDB provides its own TLS encryption, by having a central tool like Istio service mesh, you can set up an easy and consistent policy where Istio automatically manages the certificate rotation.

This tutorial focuses on how to deploy YugabyteDB with Istio mTLS to secure communication between services.

Getting started

While you can run this setup on minikube on your local machine, we will use Google Cloud and Google Kubernetes Engine (GKE) for this blog.

Prerequisites

The YugabyteDB Helm chart has been tested with the following software versions:

  • Kubernetes 1.14+ with nodes such that a total of 12 CPU cores and 45 GB RAM can be allocated to YugabyteDB. This can be three nodes with 4 CPU cores and 15 GB RAM allocated to YugabyteDB.
  • Helm 3.0 or later.
  • YugabyteDB Docker image (yugabytedb/yugabyte) 2.3.0 or later.
  • For optimal performance, ensure you’ve set the appropriate system limits using ulimit on each node in your Kubernetes cluster.
  • Istio 1.6 or later.

Prep your cluster with Istio and Kiali

Go to the Istio release page to download the installation file for your OS, or download and extract the latest release automatically (Linux or macOS):

$ curl -L https://istio.io/downloadIstio | sh -
$ cd istio-1.7.3
$ export PATH=$PWD/bin:$PATH

For this installation, we use the demo configuration profile. It’s selected to have a good set of defaults for testing.

$ istioctl install --set profile=demo
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation complete

Install Kiali for visualizing the mesh traffic.

kubectl apply -f istio-1.7.3/samples/addons/prometheus.yaml -n istio-system
kubectl apply -f istio-1.7.3/samples/addons/kiali.yaml -n istio-system

Deploy a YugabyteDB cluster via Helm

kubectl create namespace yb-demo

# Enable the automatic envoy sidecar injection on the Yugabyte namespace
kubectl label namespace yb-demo istio-injection=enabled

Install YugabyteDB using the helm charts.

helm repo add yugabytedb https://charts.yugabyte.com
helm repo update
helm search repo yugabytedb/yugabyte

Output
NAME               	CHART VERSION	APP VERSION	DESCRIPTION                                       
yugabytedb/yugabyte	2.3.2        	2.3.2.0-b37	YugabyteDB


# Set helm override to enable istio compatibility with YugabyteDB
helm install yb-demo yugabytedb/yugabyte --namespace yb-demo --set istioCompatibility.enabled=true

Ensure all pods are in a healthy and ready state.

kubectl get pods -n yb-demo
NAME           READY   STATUS    RESTARTS   AGE
yb-master-0    3/3     Running   3          18d
yb-master-1    3/3     Running   3          18d
yb-master-2    3/3     Running   3          18d
yb-tserver-0   3/3     Running   8          18d
yb-tserver-1   3/3     Running   8          18d
yb-tserver-2   3/3     Running   7          18d

Peer authentication policy

PeerAuthentication policies are used to secure service to service communication by automating the process of generation, distribution, and rotation of certificates and keys.

In Istio, peer authentication policies have three levels of granularity through which we can define our mTLS settings. For each service, Istio applies the narrowest matching policy. The order is service-specific, namespace-wide, mesh-wide. In this demo, we will be using the namespace-wide policy.

The namespace-wide PeerAuthentication policy affects all services in a namespace. The following MeshPolicy sets the mTLS mode of all services to STRICT (workloads only accept mutual TLS traffic).

Create PeerAuthentication to enforce mTLS for the yb-demo namespace.

kubectl apply -n yb-demo -f - <<EOF
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: mtls
spec:
  mtls:
    mode: STRICT
EOF

You can set mTLS to PERMISSIVE which can accept both encrypted and plain-text traffic.

 - mtls:
      mode: PERMISSIVE

The PERMISSIVE mode is particularly useful when migrating to Istio, when there are still services that are not managed by Istio (or mTLS). With the PERMISSIVE setting, these out-of-mesh services can also communicate with the services that are already in the mesh, aiding the Istio migration process.

DestinationRule

Use a DestinatonRule object to instruct client services to make a mutual TLS connection with a target service using the required certificates. DestinationRule objects configure what happens to the traffic meant for a given destination or target service.

Set the tls: mode key of the trafficPolicy section in a DestinationRule to enable or disable the TLS connection to a destination service using one of these settings: SIMPLE, MUTUAL, ISTIO_MUTUAL, or DISABLE.

We will create DestinationRule objects with ISTIO_MUTUAL mode for both yb-master and yb-tserver services.

kubectl apply -n yb-demo -f - <<EOF
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: yb-masters-mtls
spec:
  host: "yb-masters.yb-demo.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
---

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: yb-tservers-mtls
spec:
  host: "yb-tservers.yb-demo.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL
EOF

Verify the authentication policy and destination rules are set.

kubectl get PeerAuthentication,DestinationRule -n yb-demo
NAME                                        AGE
peerauthentication.security.istio.io/mtls   18d
 
NAME                                                   HOST                                    AGE
destinationrule.networking.istio.io/yb-masters-mtls    yb-masters.yb-demo.svc.cluster.local    17d
destinationrule.networking.istio.io/yb-tservers-mtls   yb-tservers.yb-demo.svc.cluster.local   17d

Deploy a sample application

We will use the yb-sample-apps to emulate various workloads against YugabyteDB.

kubectl run yb-sample-apps -it --rm  --image yugabytedb/yb-sample-apps -n yb-demo --command -- sh



java -jar yb-sample-apps.jar java-client-sql \
   --workload SqlInserts \
   --nodes yb-tserver-service.yb-demo.svc.cluster.local:5433 \
   --num_threads_write 4 \
   --num_threads_read 6

Use Kiali to visualize mesh traffic

At this point, our cluster is running a sample workload and mTLS is enabled for that namespace, so now let’s use the Kiali UI for visualizing the mesh traffic. Type the following command in a different terminal window to launch Kiali.

istioctl dashboard kiali

After you are in the Kiali UI, explore the visualizations in the navigation pane. The focus of this tutorial is on the graph view, where you can see the topology of the YugabyteDB microservices.

Open that graph view and select yb-demo in the Namespace field, which matches the Kubernetes namespace used in this tutorial. Enable the option in the Display dropdown, select the Security badge and Traffic Animation. You should see a view similar to this example:

See security in the graph UI Istio YugabyteDB Kiali example

Kiali has a useful security layer in the graph where the edge shows a lock icon for mTLS connections, and at a glance, I can confirm that mTLS is enabled between the internal YugabyteDB master and tserver services as well as for the sample application connecting to the yb-tserver-service.

Summary

This tutorial discussed how mutual TLS authentication works for YugabyteDB within the Istio service mesh environment. YugabyteDB’s cloud native and developer friendly architecture makes it a perfect fit for Kubernetes-based orchestration by seamlessly integrating within the Kubernetes ecosystem. Now you can focus on building your modern apps backed by a distributed database while offloading security concerns to Istio.

Have additional questions about YugabyteDB or mTLS? Reach out in our Slack channel and we’d love to have a conversation about it!

What’s next?

Related Posts