Thoughts on distributed databases, open source, and cloud native
Editor’s note: This post was originally published August 2, 2018 and has been updated as of May 26, 2020.
Whether it be a WordPress website’s MySQL backend or Dropbox’s multi-exabyte storage system, data replication is at the heart of making data durable and available in the presence of hardware failures such as machine crashes,
…
Read MoreOur post 6 Signs You Might be Misunderstanding ACID Transactions in Distributed Databases describes the key challenges involved in building high performance distributed transactions. Multiple open source ACID-compliant distributed databases have started building such transactions by taking inspiration from research papers published by Google. In this post, we dive deeper into Percolator and Spanner, the two Google systems behind those papers,
…
Read MoreACID transactions were a big deal when first introduced formally in the 1980s in monolithic SQL databases such as Oracle and IBM DB2. Popular distributed NoSQL databases of the past decade including Amazon DynamoDB and Apache Cassandra initially focused on “big data” use cases that did not require such guarantees and hence avoided implementing them altogether. However, ACID transactions have made a strong comeback in the last several years with the launch of next-generation distributed databases that have built-in support for them.
…
Read MoreEditor’s note – This is the first part of a two-part series on database storage engines:
When evaluating operational databases,
…
Read MoreIn the first post of this two-part series, we learned about the B-tree vs LSM approach to index management in operational databases. While the indexing algorithm plays a fundamental role in determining the type of storage engine needed, advanced considerations highlighted below are equally important to take into account.
Monolithic databases,
…
Read MoreResults from the 2018 Kubernetes Application Usage Survey should put to rest concerns enterprise users have had around the viability of Docker containers and Kubernetes orchestration for running stateful services such as databases and message queues. Its exciting to see that nearly 40% of respondents are running databases (SQL and/or NoSQL) using Kubernetes.
…
Read More
Modern user-facing apps, like E-Commerce and SaaS, frequently require features from multiple databases (broadly — SQL, NoSQL and a cache) to support their multi-workload needs. App developers are responsible for understanding and managing which pieces of data should be stored in which SQL and NoSQL database. Furthermore, the app is also responsible for moving data across the tiers (e.g.
…
Read More
ACID transactions are a fundamental building block when developing business-critical, user-facing applications. They simplify the complex task of ensuring data integrity while supporting highly concurrent operations. While they are taken for granted in monolithic SQL/relational databases, distributed NoSQL/non-relational databases either forsake them completely or support only a highly restrictive single-row flavor (see sections below). This loss of ACID properties is usually justified with a gain in performance (measured in terms of low latency and/or high throughput).
…
Read More
Updated April 2019.
The famed CAP Theorem has been a source of much debate among distributed systems engineers. Those of us building distributed databases are often asked how we deal with it. In this post, we dive deeper into the consistency-availability tradeoff imposed by CAP which is only applicable during failure conditions.
…
Read More
Today, we are launching Yugabyte out of stealth and announcing the availability of YugabyteDB’s first public beta release. Yugabyte offers an open-source, cloud-native database for mission-critical applications. Yuga in Sanskrit represents an era or an epoch (about 4.32 million human years), a very long period of time. We picked the name Yugabyte to signify data that lives forever without limits.
…
Read More
