Service Mesh Era: Building Modern Apps with YugabyteDB and Istio
Microservices architectures are becoming the de facto way developers are thinking about how their applications are constructed. But security remains a top concern for many organizations. Given the general trends of the proliferation of threats within the production network and the increased points of privileged access, it is increasingly necessary to adopt a zero-trust network security approach for microservices architectures.
One of the most common security approaches is to set up mTLS. While this is an important security tool, it’s often difficult and time consuming to manage. To start, you have to create, distribute, and rotate keys and certificates to a large number of services. You then need to ensure you are properly implementing mTLS on all of your clients and servers. One of the compelling features of Istio is the ability to uniformly administer mTLS for all of your services without sacrificing developer productivity. While it’s true YugabyteDB provides its own TLS encryption, by having a central tool like Istio service mesh, you can set up an easy and consistent policy where Istio automatically manages the certificate rotation.
This tutorial focuses on how to deploy YugabyteDB with Istio mTLS to secure communication between services.
While you can run this setup on minikube on your local machine, we will use Google Cloud and Google Kubernetes Engine (GKE) for this blog.
The YugabyteDB Helm chart has been tested with the following software versions:
- Kubernetes 1.14+ with nodes such that a total of 12 CPU cores and 45 GB RAM can be allocated to YugabyteDB. This can be three nodes with 4 CPU cores and 15 GB RAM allocated to YugabyteDB.
- Helm 3.0 or later.
- YugabyteDB Docker image (yugabytedb/yugabyte) 2.3.0 or later.
- For optimal performance, ensure you’ve set the appropriate system limits using ulimit on each node in your Kubernetes cluster.
- Istio 1.6 or later.
Prep your cluster with Istio and Kiali
Go to the Istio release page to download the installation file for your OS, or download and extract the latest release automatically (Linux or macOS):
$ curl -L https://istio.io/downloadIstio | sh - $ cd istio-1.7.3 $ export PATH=$PWD/bin:$PATH
For this installation, we use the demo configuration profile. It’s selected to have a good set of defaults for testing.
$ istioctl install --set profile=demo ✔ Istio core installed ✔ Istiod installed ✔ Egress gateways installed ✔ Ingress gateways installed ✔ Installation complete
Install Kiali for visualizing the mesh traffic.
kubectl apply -f istio-1.7.3/samples/addons/prometheus.yaml -n istio-system kubectl apply -f istio-1.7.3/samples/addons/kiali.yaml -n istio-system
Deploy a YugabyteDB cluster via Helm
kubectl create namespace yb-demo # Enable the automatic envoy sidecar injection on the Yugabyte namespace kubectl label namespace yb-demo istio-injection=enabled
Install YugabyteDB using the helm charts.
helm repo add yugabytedb https://charts.yugabyte.com helm repo update helm search repo yugabytedb/yugabyte Output NAME CHART VERSION APP VERSION DESCRIPTION yugabytedb/yugabyte 2.3.2 22.214.171.124-b37 YugabyteDB # Set helm override to enable istio compatibility with YugabyteDB helm install yb-demo yugabytedb/yugabyte --namespace yb-demo --set istioCompatibility.enabled=true
Ensure all pods are in a healthy and ready state.
kubectl get pods -n yb-demo NAME READY STATUS RESTARTS AGE yb-master-0 3/3 Running 3 18d yb-master-1 3/3 Running 3 18d yb-master-2 3/3 Running 3 18d yb-tserver-0 3/3 Running 8 18d yb-tserver-1 3/3 Running 8 18d yb-tserver-2 3/3 Running 7 18d
Peer authentication policy
PeerAuthentication policies are used to secure service to service communication by automating the process of generation, distribution, and rotation of certificates and keys.
In Istio, peer authentication policies have three levels of granularity through which we can define our mTLS settings. For each service, Istio applies the narrowest matching policy. The order is service-specific, namespace-wide, mesh-wide. In this demo, we will be using the namespace-wide policy.
The namespace-wide PeerAuthentication policy affects all services in a namespace. The following
MeshPolicy sets the mTLS mode of all services to
STRICT (workloads only accept mutual TLS traffic).
Create PeerAuthentication to enforce mTLS for the yb-demo namespace.
kubectl apply -n yb-demo -f - <<EOF apiVersion: security.istio.io/v1beta1 kind: PeerAuthentication metadata: name: mtls spec: mtls: mode: STRICT EOF
You can set mTLS to
PERMISSIVE which can accept both encrypted and plain-text traffic.
- mtls: mode: PERMISSIVE
PERMISSIVE mode is particularly useful when migrating to Istio, when there are still services that are not managed by Istio (or mTLS). With the
PERMISSIVE setting, these out-of-mesh services can also communicate with the services that are already in the mesh, aiding the Istio migration process.
Use a DestinatonRule object to instruct client services to make a mutual TLS connection with a target service using the required certificates. DestinationRule objects configure what happens to the traffic meant for a given destination or target service.
tls: mode key of the trafficPolicy section in a DestinationRule to enable or disable the TLS connection to a destination service using one of these settings:
We will create DestinationRule objects with
ISTIO_MUTUAL mode for both yb-master and yb-tserver services.
kubectl apply -n yb-demo -f - <<EOF apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: yb-masters-mtls spec: host: "yb-masters.yb-demo.svc.cluster.local" trafficPolicy: tls: mode: ISTIO_MUTUAL --- apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: name: yb-tservers-mtls spec: host: "yb-tservers.yb-demo.svc.cluster.local" trafficPolicy: tls: mode: ISTIO_MUTUAL EOF
Verify the authentication policy and destination rules are set.
kubectl get PeerAuthentication,DestinationRule -n yb-demo NAME AGE peerauthentication.security.istio.io/mtls 18d NAME HOST AGE destinationrule.networking.istio.io/yb-masters-mtls yb-masters.yb-demo.svc.cluster.local 17d destinationrule.networking.istio.io/yb-tservers-mtls yb-tservers.yb-demo.svc.cluster.local 17d
Deploy a sample application
We will use the yb-sample-apps to emulate various workloads against YugabyteDB.
kubectl run yb-sample-apps -it --rm --image yugabytedb/yb-sample-apps -n yb-demo --command -- sh java -jar yb-sample-apps.jar java-client-sql \ --workload SqlInserts \ --nodes yb-tserver-service.yb-demo.svc.cluster.local:5433 \ --num_threads_write 4 \ --num_threads_read 6
Use Kiali to visualize mesh traffic
At this point, our cluster is running a sample workload and mTLS is enabled for that namespace, so now let’s use the Kiali UI for visualizing the mesh traffic. Type the following command in a different terminal window to launch Kiali.
istioctl dashboard kiali
After you are in the Kiali UI, explore the visualizations in the navigation pane. The focus of this tutorial is on the graph view, where you can see the topology of the YugabyteDB microservices.
Open that graph view and select yb-demo in the Namespace field, which matches the Kubernetes namespace used in this tutorial. Enable the option in the Display dropdown, select the Security badge and Traffic Animation. You should see a view similar to this example:
Kiali has a useful security layer in the graph where the edge shows a lock icon for mTLS connections, and at a glance, I can confirm that mTLS is enabled between the internal YugabyteDB master and tserver services as well as for the sample application connecting to the yb-tserver-service.
This tutorial discussed how mutual TLS authentication works for YugabyteDB within the Istio service mesh environment. YugabyteDB’s cloud native and developer friendly architecture makes it a perfect fit for Kubernetes-based orchestration by seamlessly integrating within the Kubernetes ecosystem. Now you can focus on building your modern apps backed by a distributed database while offloading security concerns to Istio.
Have additional questions about YugabyteDB or mTLS? Reach out in our Slack channel and we’d love to have a conversation about it!