YugaByte Database Engineering Update – August 20, 2018
Time for another update from the engineering team at YugaByte! It has been a month since the last update, so let’s dive right in.
On the conference front, YugaByte was at Google Next towards the end of July. YugaByte DB was already very well integrated into the Google Cloud Compute ecosystem, we additionally announced support for Google Container Engine (GKE). You can also read about how YugaByte DB compares with the various Google Cloud databases.
There have been a number of questions about how YugaByte DB compares to Google Spanner. While both are geo-distributed databases that support distributed transactions, YugaByte DB is:
- Redis, Cassandra and PostgreSQL compatible APIs for reading and writing data
- Open source under Apache 2.0 license
- Supports Google Cloud, AWS, Azure, Kubernetes and private data centers
- Sept 19: “Building Modern Apps at Cloud Scale with Data Services” – Webinar
- Sept 24-27: SpringOne Platform 2018 – Washington, D.C.
- Oct 9: Microservices Day – TBD
- Oct 15-16: PostgreSQL Conf Silicon Valley 2018 – San Jose, CA
- Nov 26-30: AWS re:Invent 2018 – Las Vegas, NV
We are Hiring!
YugaByte is looking for a passionate Developer Advocate! Are you excited about becoming the voice of our users? Do you love experimenting with new technologies and presenting it at conferences, meetups and workshops? We would love to talk to you. Check out our list of open positions.
Cassandra-compatible YCQL Features
- Added support for “broadcast address” to enable public and private IP support
- Ability to return fine-grained errors in CAS (compare and set) and batch statements
- Support for weak secondary indexes for higher ingest performance
Redis-compatible YEDIS Features
- Authentication support using passwords, including support for multiple active passwords to enable online rotation of passwords. Read more.
- Safety flags to disable dropping data inadvertently, for example using commands such as FLUSHALL
- Monitor support
Ecosystem and Deployments
- Basic helm chart support to deploy Kubernetes clusters with ease.
YugaByte DB 1.1 is right around the corner! We are going to be announcing a number of critical features including:
- General availability of Distributed ACID transactions, consistent secondary indexes and a native JSON data type.
- On the security front, authentication of users in the Cassandra and Redis APIs, as well as support for TLS encryption.
- EE features such as read replica support and support for managed Kubernetes environments such as GKE and PKS are going to preview as a beta features
To view a list of all items being worked, browse to our GitHub projects page. And stay tuned for an update on our Jepsen testing soon.
Documentation, Blogs, Tutorials
We have recently added docs on how to use YugaByte DB with client drivers in C++, C# and Go in addition to the existing languages such as NodeJS and Java. Happy app-building!
Check out how to use the MONITOR command in the YEDIS API that is a great debugging tool to see all requests being processed by a server.
We have been busy on the blogging front with a bunch of tech deep-dive posts.
- Read about how we tried to torture our own database by running a high density workload on YugaByte DB. We loaded 18TB of data on a 4 node AWS cluster, ran a brutal read workload on it and finally added a node to scale it out!
- Want to use consistent secondary indexes with a NoSQL DB? No problem, this post will walk you through using secondary indexes in YugaByte DB.
- There is a lot of talk about how Raft is great. Now you can take a look under the hood and understand how Raft is really used in YugaByte DB.
- Looking to understand how to run a distributed database on Kubernetes? Our post on understanding how YugaByte DB runs on Kubernetes walks you through all the details including instructions on how to run your own YugaByte DB cluster on minikube.
- You can also read all about the architecture of Apache Cassandra, and some core architectural differences when comparing with YugaByte DB.
Once again, there has been no dearth of enhancement requests. There are too many to list, but here are three useful enhancements that we have recently implemented.
Broadcast address support in Cassandra API
In modern cloud-based deployments, servers could have public IP address or DNS names that they are not aware of. A cloud provider could provide public ip address functionality via port forwarding, for example. Also clouds such as GCP and AWS have different billing strategies when traffic goes through public IP vs private IP. In many of these scenarios, the services on these servers know nothing about the DNS names (or public IP addresses).
As an example, consider machines instances on most public clouds. They usually have a public IP address that should be used from outside the cloud region and a private IP address that is preferred inside. This feature allows YugaByte DB to span regions or cloud providers by allowing the appropriate address to be used for inter-server communication.
Multiple databases in the Redis API
In many scenarios, it is very desirable to allow applications to interact with only their data (and not mix data across apps together). This feature enables this by allowing creation of multiple “databases” that can be used by different applications. Open-source Redis already allows for multiple databases, but this is generally not the recommended practice due to the single-threaded nature of the server. However, with YugaByte DB offering a persistent and elastic Redis database across multiple nodes, this feature was highly requested by many of our users.
In YugaByte DB, each redis database is modeled as a separate DocDB table. The database name can be any arbitrary string (it has to be an integer in open-source Redis). New commands have been added to the YEDIS API – CreateDB, ListDB, DeleteDB and Select to enable working with databases. You can read more details about the design of databases in Yedis.
Authentication support with multiple passwords in Redis API
Security is a critical part of databases. To secure the Redis API on YugaByte DB deployments, we enhanced the Redis AUTH command to support authentication for client connections by storing credentials in a centralized location in the DB. We got an enhancement request to support authentication of clients against multiple passwords simultaneously. This enables the app tier to be upgraded in a rolling fashion, because during the upgrades, some clients would use the old password while others would use the new one. This feature enables a zero-downtime upgrade of the application tier while not compromising on security.