The Distributed SQL Blog

Thoughts on distributed databases, open source and cloud native

Automating YugabyteDB Deployments with AWS CloudFormation

YugabyteDB is easy to get started with on the infrastructure of your choice including public cloud platforms, private cloud environments, and any Kubernetes distribution. For example, you can quickly customize and deploy in AWS thanks to CloudFormation templates. AWS CloudFormation is one of the many ways to automate a public cloud deployment in a consistent manner.

Before we dive in, we wanted to let you know that the Yugabyte team has been working from home in order to do our part with social distancing and to help with containment efforts. We have also transitioned to online meetings with our customers, partners, candidates, community, and fellow Yugabeings. We will continue with these and other measures until it’s determined to be safe otherwise. We wish everyone the best health and wellness, and our thoughts are with all those around the world who have been affected by COVID-19. If during these unprecedented circumstances, you have some time or would welcome a brief distraction, we invite you to check out this post below.

A principal concern for service availability is the capability to survive a given physical outage. Public clouds are designed with availability zones (AZs) so that customers can recognize and architect solutions that will have high availability even with an outage of an entire AZ. A legacy SQL database would need special add-on software components and most likely be re-architected to leverage a multi-AZ active-active or active-passive scenario. YugabyteDB’s distributed SQL architecture is specifically designed for resilience, scalability, and geo-distribution by default in dynamic infrastructure environments such as VMs and containers. Using AWS CloudFormation as a template, you can easily automate YugabyteDB deployments in your own AWS account.

YugabyteDB CloudFormation Configuration

AWS CloudFormation templates are stored conveniently in plain-text YAML format. If you open the sample YAML file in a text editor, you can review the defaults and proposed configuration:

  • YugabyteDB Version: YugabyteDB’s template should be deployed with our latest software version which downloads and installs from our download.yugabyte.com repo.
  • Replication Factor (RF): This is set to 3 which means every tablet (aka shard) will have 3 replicas. In order to ensure fault tolerance against node failures, this setting will also lead to the creation of 3 AWS EC2 nodes with one replica placed on each node.
  • KeyName: AWS assigns keypairs for accessing individual nodes in AWS public cloud infrastructure. You can review how to setup keypairs here.
  • LatestAmiId: YugabyteDB is built on Linux and our template helpfully uses a reference to deploy the latest version of Amazon Linux 2.
  • VPC: This CloudFormation template will create its own VPC to host the YugabyteDB cluster. Ensure you have capacity to create an additional VPC within your AWS quota. Customers can request a quota increase if necessary. If you cannot create a new VPC, then make sure to edit the template to use an existing VPC.
  • YugaByteNodeSG: YugabyteDB nodes will be deployed with a consistent security group configuration across 3 AZ’s and public subnets. This security group will open the database server RPC ports (7100 and 9100) and the database server API ports (5433, 6379, and 9042) internally across the subnets. It will also open the management ports (22, 7000, and 9000) external to the subnets. A list of default ports used on a YugabyteDB cluster are available here.
  • BlockDeviceMappings: Each YugabyteDB node will have a 50GB EBS volume attached. This can be customized for larger storage. For AWS instances with faster storage such as i3 and NVMe storage, you can format and use this storage as a disk for your database files. YugabyteDB’s i3 example is here.

Now that we’ve stepped through the essential components of our configuration, let’s see how easy it is to use this in practice.

Deployment Steps

Prerequisites

Steps

[1] Download the AWS CloudFormation YAML file as shown below.

[2] Login to your AWS console and go to the CloudFormation UI.

YugabyteDB and AWS CloudFormation How To, log into AWS console and go to CloudFormation UI

[3] Click on “Create Stack”.

YugabyteDB and AWS CloudFormations How To, create a stack

[4] To prepare a template, upload the “yugabyte_cloudformation.yaml” file from the repo directory.

YugabyteDB and AWS CloudFormations distributed sql how to template upload step 4

[5] Set a CloudFormation stack name and confirm the YugabyteDB version, instance size, user keypair (for interactive SSH login), AMI-ID, and replication factor. The example YAML file will have some of this information set already. If you would like a different instance size or base linux OS, you can customize that within the YAML file and re-upload.

YugabyteDB distributed SQL db and AWS CloudFormation how to step 5

[6] You can choose to customize any additional stack options at this point, but YugabyteDB has enough information to complete its deployment.

Customize stack options YugabyteDB distributed sql database using AWS CloudFormation How To

[7] Before deployment, you can review all of your setup options as well as open another browser window to the AWS cost estimate.

view step 7 of YugabyteDB using AWS CloudFormations How To, review setup options

Specify stack details YugabyteDB distributed SQL database using AWS CloudFormations

[8] You can view the CloudFormation deployment progress in the UI and refresh as necessary.

view the CloudFormation deployment progress YugabyteDB AWS CloudFormation

[9] Once CloudFormation is finished creating and starting up the individual virtual machines, you can confirm the specific instance information in the EC2 UI.

Step 9 YugabyteDB distributed sql db and AWS CloudFormation How To - confirm the specific instance information in the EC2 UI

[10] When you click on any of the nodes, you can see that they are part of the same YugaByteNodeSG.

Nodes part of the same YugaByteNodeSG - YugabyteDB distributed sql and AWS CloudFormation

Step 10 YugabyteDb distributed SQL using AWS CloudFormation

[11] If you want to customize the security group for accessing the YSQL client port, you can update the YugaByteNodeSG in place.

customize the security group for accessing the YSQL client port YugabyteDB using AWS CloudFormation How To

[12] You can log in to a given node using the public node IP as well as the keypair credentials you specified during the CloudFormation template setup.

Step 12 YugabyteDB distributed SQL db using CloudFormation how-to

[13] Now that you’re up and running with YugabyteDB in multiple AZ’s, you can use any of our other examples to test the distributed SQL database. For example, you could upload and experiment with the sample Northwind database.

[14] Once you are ready, it is easy to deprovision the YugabyteDB cluster. Go back to the CloudFormation UI, select the stack, and click on “Delete”.

easy to deprovision the YugabyteDB cluster AWS CloudFormation distributed SQL how to

easy to deprovision yugabytedb distributed sql db using aws cloudformation

delete in progress yugabytedb aws cloudformation howto

TLDR

As a simpler option, if you would like to simply try a “one-click” approach to deploying YugabyteDB with CloudFormation, there is a link on our GitHub repo that launches you directly into the AWS CloudFormation template.

“one-click” approach to deploying YugabyteDB with CloudFormation

When you click on “Launch Stack” you will be redirected to your AWS console. After logging in with your AWS account and selecting your region, you will have easy access to all of the configuration options on a single page.

easy access to the configuration options on a single page yugabytedb cloudformations aws

configuration options yugabytedb aws cloudformation how to

Conclusion

That’s all there is to creating a simple YugabyteDB deployment in AWS. Spinning up a scalable and resilient distributed SQL database for cloud native application development has never been easier. The example shown here can be re-used to create multiple clusters or to quickly provision and deprovision clusters for additional testing. You can also customize and update your deployment configuration using the provided YAML file as a starting point.

Related Posts