The Distributed SQL Blog

Thoughts on distributed databases, open source and cloud native

YSQL Architecture: Implementing Distributed SQL in YugaByte DB

In this post, we will look at the architecture of YSQL, the PostgreSQL-compatible distributed SQL API in YugaByte DB. We will also touch on the current state of the project and the next steps in progress. Here is a quick overview:

  • YugaByte DB has a common distributed storage engine that powers both SQL and NoSQL
  • For supporting NoSQL apps,

Read More

Apache Cassandra: The Truth Behind Tunable Consistency, Lightweight Transactions & Secondary Indexes

ACID transactions were a big deal when first introduced formally in the 1980s in monolithic SQL databases such as Oracle and IBM DB2. Popular distributed NoSQL databases of the past decade including Apache Cassandra initially focused on “big data” use cases that did not require such guarantees and hence avoided implementing them altogether. Our post, “A Primer on ACID Transactions: The Basics Every Cloud App Developer Must Know”

Read More

Google Spanner vs. Calvin: Is There a Clear Winner in the Battle for Global Consistency at Scale?

Prof. Daniel Abadi, lead inventor of the Calvin transaction management protocol and the PACELC theorem, wrote a thought-provoking post last month titled “NewSQL database systems are failing to guarantee consistency, and I blame Spanner”. The post takes a negative view of software-only Google Spanner derivative databases such as YugaByte DB and CockroachDB that use Spanner-like partitioned consensus for single shard transactions and a two phase commit (2PC) protocol for multi-shard (aka distributed) ACID transactions.

Read More

YugaByte DB 1.1 New Feature: Public IPs to Simplify Multi or Hybrid Cloud Database Deployments

Welcome to another post in our ongoing series that highlights new features from the latest 1.1 release announced last week. Today we are going to look at the importance of public IP addresses and hostnames in simplifying multi-cloud and hybrid cloud deployments.

In modern cloud deployments, servers often have a combination of private IP addresses (used in the private LAN and often the IP address of the network interface on the server),

Read More

YugaByte DB 1.1 New Feature: Document Data Modeling with the JSON Data Type

Welcome to another post in our ongoing series that highlights new features from the latest 1.1 release announced last week. Today we are going to look at document data modeling using the native JSON data type available in YugaByte DB’s Cassandra compatible YCQL API. Note that this data type is specific to YugaByte DB and is not part of the standard Cassandra Query Language (CQL).

Read More

YugaByte DB 1.1 New Feature: Speeding Up Queries with Secondary Indexes

Welcome to another post from our ongoing series where we highlight a new feature from the latest 1.1 release! Today we are going to look at secondary indexes.

Defining Secondary Indexes

A database index is a data structure that improves the speed of data retrieval operations on a database table. Typically, databases are very efficient at looking up data by the primary key.

Read More

Jepsen Testing on YugaByte DB

At YugaByte, our mission is to build a robust, reliable, distributed OLTP database. Needless to say, we take correctness and technical accuracy of our claims very seriously. Therefore, we absolutely love a testing framework like Jepsen which helps verify correctness and are fans of Kyle Kingsbury’s work!

Here is a summary of what we have done so far in regards to Jepsen:

  • We have performed our own DIY style Jepsen testing
  • The YugaByte DB Jepsen testing repository is open source
  • For the current suite of Jepsen tests for YugaByte DB that we have tested in a loop,

Read More

How Does the Raft Consensus-Based Replication Protocol Work in YugaByte DB?

As we saw in ”How Does Consensus-Based Replication Work in Distributed Databases?”, Raft has become the consensus replication algorithm of choice when it comes to building resilient, strongly consistent systems. YugaByte DB uses Raft for both leader election and data replication. Instead of having a single Raft group for the entire dataset in the cluster,

Read More

How Does Consensus-Based Replication Work in Distributed Databases?

Whether it be a WordPress website’s MySQL backend or Dropbox’s multi-exabyte storage system, data replication is at the heart of making data durable and available in the presence of hardware failures such as machine crashes, disk failures, network partitions and clock skews. The basic idea behind replication is very simple: keep multiple copies of data on physically isolated hardware so that the failure in one does not impact the others and as a result,

Read More