Designing Data-Intensive Applications The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

Martin Kleppmann — 2017

Data is at the center of many challenges in system design today. Difficult issues need to be figured out, such as scalability, consistency, reliability, efficiency, and maintainability. In addition, we have an overwhelming variety of tools, including relational databases, NoSQL datastores, stream or batch processors, and message brokers. What are the right choices for your application? How do you make sense of all these buzzwords? In this practical and comprehensive guide, author Martin Kleppmann helps you navigate this diverse landscape by examining the pros and cons of various technologies for processing and storing data. Software keeps changing, but the fundamental principles remain the same. With this book, software engineers and architects will learn how to apply those ideas in practice, and how to make full use of data in modern applications. Peer under the hood of the systems you already use, and learn how to use and operate them more effectively Make informed decisions by identifying the strengths and weaknesses of different tools Navigate the trade-offs around consistency, scalability, fault tolerance, and complexity Understand the distributed systems research upon which modern databases are built Peek behind the scenes of major online services, and learn from their architectures

Reviews

Bouke van der Bijl@bouk

Mar 1, 2023

This book is incredibly useful if you spend time engineering software systems that process data (AKA all systems). In my career at Shopify I've spent time as a Data Engineer working on Hadoop/Spark batch processing systems, on the 'Merchant Analytics' team working on a stream processing and real-time analytics database, and as an application developer working on standard Rails apps. Having this book available to me would've been incredibly useful, and it contains many many lessons I've learned over the past couple of years, but written down in a way that's way clearer than I could ever formulate it.

Bruce Wang@number5

Jan 12, 2023

A book explained all the modern database-like techs and how their internal works, insightful yet easy to understand, recommended for everyone who write code

Hatem Abdelmowgoud @iammowgoud

Aug 31, 2022

This book is definitely a classic. The book basically touches upon the surface of a wide array of topics related to handling data in a distributed environment; ranging from basic database theory, ACID, replication and partitioning to more complex (and "modern") topics like stream and batch processing on the cloud. Martin Kleppman lays down all the needed information on what every engineer needs to know about designing systems that deal with any kind of data. Although the book doesn't dive into deep technical details, it has a very good bibliography and footnotes that leads you to all of the academic papers you need. Overall, the book is an essential read for any software/data engineer in 2019.