The Distributed SQL Blog

Thoughts on distributed databases, open source and cloud native

5 Reasons Why Apache Kafka Needs a Distributed SQL Database

Modern enterprise applications must be super-elastic, adaptable, and running 24/7. However, traditional request-driven architectures entail a tight coupling of applications. For example, App 1 asks for some information from App 2 and waits. App 2 then sends the requested information to App 1. This sort of app-to-app coupling hinders development agility and blocks rapid scaling.

In event-driven architectures,

Read More

Achieving Fast Failovers After Network Partitions in a Distributed SQL Database

In February of this year, Kyle Kingsbury of Jepsen.io was conducting formal testing of YugaByte DB for correctness under extreme and unorthodox conditions. Obviously, simulating all manner of network partitions is part of his testing methodology. As a result, during his testing he spotted the fact that although nodes would reliably come back after a failure,

Read More

6 Technical Challenges Developing a Distributed SQL Database

You can join the discussion on HackerNews here.

We crossed the three year mark of developing YugabyteDB in February of this year. It has been a thrilling journey thus far, but not without its fair share of technical challenges. There were times when we had to go back to the drawing board and even sift through academic research to find a better solution than what we had at hand.

Read More

How to Achieve High Availability, Low Latency & GDPR Compliance in a Distributed SQL Database

Today’s developers understand that the key requirement to converting and retaining customers is all about delivering fast and responsive experiences, while remaining resilient to failures and compliant with data governance regulations. YugaByte DB is purpose built for geo-distributed applications that require high availability, high performance and regulatory compliance. In this blog, we are going to “look under the hood,” to explore exactly how YugaByte DB distributes data across multiple clouds,

Read More

YugabyteDB 1.2 Passes Jepsen Testing

You can join the discussion about the results on HackerNews here.

Last year we published our DIY Jepsen testing results – including the tests and failure modes implemented as well as the bugs found. We recently engaged Kyle Kingsbury, the creator of the Jepsen test suite, for an official analysis and are happy to report that YugabyteDB 1.2 formally passes Jepsen tests using the YCQL API.

Read More

Distributed PostgreSQL on a Google Spanner Architecture – Query Layer

Our previous post dived into the details of the storage layer of YugabyteDB called DocDB, a distributed document store inspired by Google Spanner. This post focuses on Yugabyte SQL (YSQL), a distributed, highly resilient, PostgreSQL-compatible SQL API layer powered by DocDB. A follow-up post will highlight the challenges faced and lessons learned when engineering such a database.

YSQL,

Read More

Distributed PostgreSQL on a Google Spanner Architecture – Storage Layer

In this post, we’ll dive into the architecture of the distributed storage layer of YugabyteDB, which is inspired by Google Spanner’s design. Our subsequent post covers the Query Layer, where the storage layer meets PostgreSQL as the SQL API. Finally, here is a follow-up post that highlights the key technical challenges we faced while engineering a distributed SQL database like YugabyteDB.

Read More

YugaByte DB High Availability & Transactions for PostgreSQL & MongoDB Developers

In the first post of our series comparing YugaByte DB with PostgreSQL and MongoDB, we mapped the core concepts in YugaByte DB to the two popular databases. This post is a deeper dive into the high availability and transactions architecture of these databases.

High Availability

Almost all databases including YugaByte DB use replication to ensure that the database remains highly available under failures.

Read More

Mapping YugabyteDB Concepts to PostgreSQL and MongoDB

If you are developing a new distributed application or are extending an existing one with a new set of microservices, chances are you are going to need to store data in a distributed SQL database. The plethora of niche databases that have emerged over the last decade make the task of selecting a database challenging. With many databases, each with its own nomenclature and nuances to choose from,

Read More

Enhancing RocksDB for Speed & Scale

This blog post was co-authored by Mikhail Bautin and Kannan Muthukkaruppan

As described in our previous post “How We Built a High Performance Document Store on RocksDB?”, YugabyteDB’s distributed document store (DocDB) uses RocksDB as its per-node storage engine. We made multiple performance and data density related enhancements to RocksDB in the course of embedding it into DocDB’s document storage layer (figure below).

Read More