The Distributed SQL Blog

Thoughts on distributed databases, open source and cloud native

Low Latency Reads in Geo-Distributed SQL with Raft Leader Leases

Note: This post contains interactive animations that explain how some of these complex algorithms work. Please view this post in a suitable media (at least 1000px by 600px screen resolution) for best results.

In this blog post, we are going to dive deep into the read performance of Raft – why read performance can take a hit and how it can be improved using leader leases.

Read More

How Data Sharding Works in a Distributed SQL Database

Enterprises of all sizes are embracing rapid modernization of user-facing applications as part of their broader digital transformation strategy. The relational database (RDBMS) infrastructure that such applications rely on suddenly needs to support much larger data sizes and transaction volumes. However, a monolithic RDBMS tends to quickly get overloaded in such scenarios. One of the most common architectures to get more performance and scalability in an RDBMS is to “shard” the data.

Read More

How to Handle Runaway Queries in a Distributed SQL Database

Runaway queries are queries that scan through a large set of data. Such queries consume vast amounts of I/O and CPU resources of the database in the background, even if the results appear as harmless timeouts to the end user or the client application. How do runaway queries get executed in the first place, anyway? Everyone who uses databases has at some point or another entered SELECT * from some_large_table,

Read More

5 Reasons Why Apache Kafka Needs a Distributed SQL Database

Modern enterprise applications must be super-elastic, adaptable, and running 24/7. However, traditional request-driven architectures entail a tight coupling of applications. For example, App 1 asks for some information from App 2 and waits. App 2 then sends the requested information to App 1. This sort of app-to-app coupling hinders development agility and blocks rapid scaling.

In event-driven architectures,

Read More

Achieving Fast Failovers After Network Partitions in a Distributed SQL Database

In February of this year, Kyle Kingsbury of Jepsen.io was conducting formal testing of YugaByte DB for correctness under extreme and unorthodox conditions. Obviously, simulating all manner of network partitions is part of his testing methodology. As a result, during his testing he spotted the fact that although nodes would reliably come back after a failure,

Read More

6 Technical Challenges Developing a Distributed SQL Database

You can join the discussion on HackerNews here.

We crossed the three year mark of developing YugaByte DB in February of this year. It has been a thrilling journey thus far, but not without its fair share of technical challenges. There were times when we had to go back to the drawing board and even sift through academic research to find a better solution than what we had at hand.

Read More

How to Achieve High Availability, Low Latency & GDPR Compliance in a Distributed SQL Database

Today’s developers understand that the key requirement to converting and retaining customers is all about delivering fast and responsive experiences, while remaining resilient to failures and compliant with data governance regulations. YugaByte DB is purpose built for geo-distributed applications that require high availability, high performance and regulatory compliance. In this blog, we are going to “look under the hood,” to explore exactly how YugaByte DB distributes data across multiple clouds,

Read More

YugaByte DB 1.2 Passes Jepsen Testing

You can join the discussion about the results on HackerNews here.

Last year we published our DIY Jepsen testing results – including the tests and failure modes implemented as well as the bugs found. We recently engaged Kyle Kingsbury, the creator of the Jepsen test suite, for an official analysis and are happy to report that YugaByte DB 1.2 formally passes Jepsen tests using the YCQL API.

Read More

Distributed PostgreSQL on a Google Spanner Architecture – Query Layer

Our previous post dived into the details of the storage layer of YugaByte DB called DocDB, a distributed document store inspired by Google Spanner. This post focuses on YugaByte SQL (YSQL), a distributed, highly resilient, PostgreSQL-compatible SQL API layer powered by DocDB. A follow-up post will highlight the challenges faced and lessons learned when engineering such a database.

Read More

Distributed PostgreSQL on a Google Spanner Architecture – Storage Layer

In this post, we’ll dive into the architecture of the distributed storage layer of YugaByte DB, which is inspired by Google Spanner’s design. Our subsequent post covers the Query Layer, where the storage layer meets PostgreSQL as the SQL API. Finally, here is a follow-up post that highlights the key technical challenges we faced while engineering a distributed SQL database like YugaByte DB.

Read More