The Distributed SQL Blog

Thoughts on distributed databases, open source and cloud native

How Data Sharding Works in a Distributed SQL Database

Enterprises of all sizes are embracing rapid modernization of user-facing applications as part of their broader digital transformation strategy. The relational database (RDBMS) infrastructure that such applications rely on suddenly needs to support much larger data sizes and transaction volumes. However, a monolithic RDBMS tends to quickly get overloaded in such scenarios. One of the most common architectures to get more performance and scalability in an RDBMS is to “shard” the data.

Read More

How to Handle Runaway Queries in a Distributed SQL Database

Runaway queries are queries that scan through a large set of data. Such queries consume vast amounts of I/O and CPU resources of the database in the background, even if the results appear as harmless timeouts to the end user or the client application. How do runaway queries get executed in the first place, anyway? Everyone who uses databases has at some point or another entered SELECT * from some_large_table,

Read More

5 Reasons Why Apache Kafka Needs a Distributed SQL Database

Modern enterprise applications must be super-elastic, adaptable, and running 24/7. However, traditional request-driven architectures entail a tight coupling of applications. For example, App 1 asks for some information from App 2 and waits. App 2 then sends the requested information to App 1. This sort of app-to-app coupling hinders development agility and blocks rapid scaling.

In event-driven architectures,

Read More

Achieving Fast Failovers After Network Partitions in a Distributed SQL Database

In February of this year, Kyle Kingsbury of Jepsen.io was conducting formal testing of YugaByte DB for correctness under extreme and unorthodox conditions. Obviously, simulating all manner of network partitions is part of his testing methodology. As a result, during his testing he spotted the fact that although nodes would reliably come back after a failure,

Read More

6 Technical Challenges Developing a Distributed SQL Database

You can join the discussion on HackerNews here.

We crossed the three year mark of developing YugaByte DB in February of this year. It has been a thrilling journey thus far, but not without its fair share of technical challenges. There were times when we had to go back to the drawing board and even sift through academic research to find a better solution than what we had at hand.

Read More

How to Achieve High Availability, Low Latency & GDPR Compliance in a Distributed SQL Database

Today’s developers understand that the key requirement to converting and retaining customers is all about delivering fast and responsive experiences, while remaining resilient to failures and compliant with data governance regulations. YugaByte DB is purpose built for geo-distributed applications that require high availability, high performance and regulatory compliance. In this blog, we are going to “look under the hood,” to explore exactly how YugaByte DB distributes data across multiple clouds,

Read More

YugaByte DB 1.2 Passes Jepsen Testing

You can join the discussion about the results on HackerNews here.

Last year we published our DIY Jepsen testing results – including the tests and failure modes implemented as well as the bugs found. We recently engaged Kyle Kingsbury, the creator of the Jepsen test suite, for an official analysis and are happy to report that YugaByte DB 1.2 formally passes Jepsen tests using the YCQL API.

Read More

Distributed PostgreSQL on a Google Spanner Architecture – Query Layer

Our previous post dived into the details of the storage layer of YugaByte DB called DocDB, a distributed document store inspired by Google Spanner. This post focuses on YugaByte SQL (YSQL), a distributed, highly resilient, PostgreSQL-compatible SQL API layer powered by DocDB. A follow-up post will highlight the challenges faced and lessons learned when engineering such a database.

Read More

Distributed PostgreSQL on a Google Spanner Architecture – Storage Layer

In this post, we’ll dive into the architecture of the distributed storage layer of YugaByte DB, which is inspired by Google Spanner’s design. Our subsequent post covers the Query Layer, where the storage layer meets PostgreSQL as the SQL API. Finally, watch for a follow-up post that highlights the lessons we learned while engineering the SQL capabilities of YugaByte DB.

Read More

YugaByte DB High Availability & Transactions for PostgreSQL & MongoDB Developers

In the first post of our series comparing YugaByte DB with PostgreSQL and MongoDB, we mapped the core concepts in YugaByte DB to the two popular databases. This post is a deeper dive into the high availability and transactions architecture of these databases.

High Availability

Almost all databases including YugaByte DB use replication to ensure that the database remains highly available under failures.

Read More