Database Performance at Scale

Chapter 1 - A Taste of What You’re Up Against: Two Tales

Chapter 2 - Your Project, Through the Lens of Database Performance

Learn More

DynamoDB Transactions Performance Testing - Alex Debrie (DeBrie Advisory) shares his assessment of how large of a performance cost is incurred with DynamoDB transactions

Chapter 3 - Database Internals: Hardware and Operation System Interactions

Learn More

Building Efficient I/O Intensive Applications with Seastar - Avi Kivity (ScyllaDB) on the theory behind Seastar, how it is used in practice, and the advanced C++ techniques used.
Different I/O Access Methods for Linux, What We Chose for ScyllaDB, and Why - Avi Kivity (ScyllaDB) shares why ScyllaDB chose asynchronous direct I/O (AIO/DIO) and explains how that decision has impacted engineering efforts as well as product performance
A discussion on readahead [LWN.net] - Jake Edge's thoughts on Linux readahead prefetching from a performance standpoint
What We Need to Unlearn About Persistent Storage - Pavel Emelyanov (ScyllaDB) talks about ways to measure the performance of modern hardware and what it all means for databases
How io_uring and eBPF Will Revolutionize Programming in Linux - Glauber Costa (Turso) shares his take on why io_uring and eBPF will completely change the way applications work with, and think about, the Linux Kernel
Linux Kernel vs DPDK: HTTP Performance Showdown - Marc Richards (AWS) uses an HTTP benchmark to compare the performance of the Linux kernel networking stack with userspace networking doing kernel bypass via DPDK
How Application-Level Priority Management Keeps Latency Low and Throughput High - Avi Kivity (ScyllaDB) on how high throughput and low latency can both be achieved in a single application by using application-level priority scheduling
Inside ScyllaDB’s Internal Cache - Tomasz Grabiec (ScyllaDB) on why it makes sense to completely bypasses the Linux cache during reads and use a specialized ,highly efficient row-based cache instead

Chapter 4 - Database Internals: Algorithmic Optimizations

Chapter 5 - Database Drivers

Learn More

We’re Porting Our Database Drivers to Async Rust - Piotr Sarna (ScyllaDB) explains the design decisions taken in implementing a Rust driver and unifying unify all other ScyllaDB drivers to use Rust underneath
Optimizing Performance in Rust for Low-Latency Database Drivers Piotr Grabowski (ScyllaDB) on the performance gains achieved by using the Tokio framework as the backbone of a new, high-performance driver and optimizing various driver components

Chapter 6 - Getting Data Closer

Learn More

The Whole WebAssembly Enchilada - Brian Sletten (author of "WebAssembly: The Definitive Guide") highlights major trends and offers some suggestions about where WebAssembly is going and why now is the time to start paying attention
What the Heck is the Edge Anyway? - Glauber Costa (Turso) on why bringing compute to the edge only solves half the problem, especially if you’re making calls to a centralized database somewhere in a far away cluster
WebAssembly Functions for your SQLite-Compatible Database - Piotr Sarna (Turso) on creating WebAssembly user-defined functions for libSQL, an open source, open contribution fork of SQLite
How WebAssembly is Eating the Database, One UDF At a Time - Dominique Saulet and Steve Manuel (dylibso) on what’s driving the new wave of UDF implementations, along with some thoughts and considerations for the future

Chapter 7 - Infrastructure & Deployment Models

Learn More

Top Mistakes with ScyllaDB: Intro & Infrastructure - Felipe Mendes applying the lessons of this chapter to a specific database, ScyllaDB.
Top Mistakes with ScyllaDB: Storage - Felipe Mendes applying the storage-focused lessons of this chapter to a specific database, ScyllaDB.

Chapter 8 - Topology Considerations

Chapter 9 - Benchmarking

Learn More

How to Measure Latency - Heinrich Hartmann (Zalando) shares strategies for avoiding pitfalls with collecting, aggregating and analyzing latency data for monitoring and benchmarking.
Solving the Issue of Mysterious Database Benchmarking Results - Daniel Seybold (benchANT) on the unique challenges of running database benchmarks in the cloud
The Do's and Don'ts of Database Benchmarking - Glauber Costa (ScyllaDB) outlines a set of best practices to improve the quality of your benchmarks
Database Benchmarking Efficiency with OtterTune’s Andy Pavlo - Andy Pavlo (Ottertune) on “databaseology” and optimizing databases for efficient testing
How to Benchmark Redis Filipe Oliveira and Or Shwartz (Redis) present several benchmark tools that you can use to understand how Redis (and other databases) performs in an independent, stable, and easily reproducible manner
Demystifying Benchmarks: How to Use Them To Better Evaluate Databases - Peter Friedenbach (Clustrix) covers different benchmarks measures, why they matter, and which ones best apply to your particular use case
Fair Benchmarking Considered Difficult: Common Pitfalls In Database Performance Testing - Mark Raasveldt, Pedro Holanda, Tim Gubner & Hannes Mühleisen share their a study of the common pitfalls in DBMS performance comparisons and how they can be spotted and avoided

Chapter 10 - Monitoring

perf Heat Maps

Learn More

How to Monitor PostgreSQL - Baron Schwartz (Percona) on how to monitor a database by understanding the difference between workload and resource monitoring—and the golden signals for each
Observability Best Practices in Distributed Databases - Felipe Cardeneti Mendes (ScyllaDB) on distributed database observability, showing several real-world situations that may affect workload performance and how to diagnose them
Top DynamoDB Performance Metrics - Jean-Mathieu Saponaro (Datadog) on key performance metrics across requests and throttling, errors, and Global Secondary Index creation
Top Cassandra Performance Metrics - John Matso (Datadog) on key performance metrics across throughput, latency, disk usage, GC, and more

Chapter 11 - Admin

How a Database Looks from a Disk’s Perspective

Appendix A - A Brief Look at Fundamental Database Design Decisions

NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence, a book by Pramod J. Sadalage and Martin Fowler
Designing Data-Intensive Applications, a book by Martin Kleppman
Database Internals, a book by Alex Petrov

Learn More

Introduction to NoSQL - Martin Fowler's crash course on NoSQL foundations
CMU Database Group's YouTube Channel - Andy Pavlo offers free online access to his much-talked-about CMU courses, which range from intro to advanced topics
Avoiding Data Hotspots at Scale - Konstantine Osipov (ScyllaDB) addresses the tradeoffs between hash and range-based sharding
Building State-of-the-Art LSM-tree compaction - Raphael Carvalho (ScyllaDB) on using RUM conjecture and controller theory to engineer state-of-art LSM-tree compaction
Scaling Raft - Konstantine Osipov (ScyllaDB) on the challenges of supporting hundreds of instances of the Raft protocol on a single node and how to significantly reduce the static burden on the network and disk each Raft group creates
Consistency Tradeoffs in Modern Distributed Database System Design - Dr. Daniel Abadi (UMD) introduces the PACELC theorem: In case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C) (as per the CAP theorem), but else (E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and consistency (C)
Leveraging Consistent Hashing in your Python Applications - Alexys Jacob (Numberly) introduces consistent hashing, and the problems it solves, by walking through a practical use case in a Python application
You're Already Eventually Consistent - Jon Haddad (Rustyrazorblade) clears up some misconceptions about eventual consistency
Inconsistent Thoughts on Database Consistency - Alex Debrie (DeBrie Advisory) discusses the various definitions of the word "consistency" that are used in the distributed databases space then shares some of his issues with conversations about consistency
Understanding Eventual Consistency in DynamoDB - Alex Debrie (DeBrie Advisory) dispels some of the fear around eventual consistency in DynamoDB

Data Modeling

Getting data modeling right is vital for database performance, but it's a massive topic that varies significantly across database models and types. We reference data modeling throughout the book, but did not feel that adding a dedicated chapter on data modeling would be feasible or adequate. Here are some resources that focus on the topic:

NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence - A book by Pramod J. Sadalage and Martin Fowler
Data Modeling Concepts and Principles - Pascal Desmarets (Hackolade) discusses the foundations of NoSQL data modeling
NoSQL Data Modeling 101 - Tzach Livatan (ScyllaDB) covers NoSQL vs SQL data modeling and how to get started with NoSQL (primary key selection, application query analysis, & more)
Top NoSQL Data Modeling Mistakes for Performance - Felipe Cardeneti Mendes (ScyllaDB) on how to avoid the top data modeling mistakes that impact performance (access the demonstrated "bad practices" in this GitHub repo)
Indexes, Filters, and Other Animals - Piotr Sarna (Turso) on the performance impacts of global/local secondary indexes and filtering
NoSQL Data Modeling Mistakes that Ruin Performance - Felipe Cardeneti Mendes explores three of the most common ways to ruin your NoSQL database performance, along with tips on how to avoid or resolve them
The Myth of Schema-less - Jon Haddad (Rustyrazorblade) debunking the myth of schemaless databases
Everything You Need to Know about DynamoDB Partitions - Alex Debrie (DeBrie Advisory) takes a deep look at DynamoDB partitions -- what they are, why they matter, and how they should affect your data modeling
The What, Why, and When of Single-Table Design with DynamoDB - Alex Debrie (DeBrie Advisory) provides a deep dive on the concepts behind DynamoDB's single-table design

Thanks Cynthia, Felipe, Piotr, and Pavel at scylladb for this awesome article.