Skip to content

Facebook's Creation of Apache Cassandra: A Technical Case Study

Technical Case Study Prompt Please generate a technical case study about Facebook's creation of Cassandra around 2008. Use the STAR process: 1. Situation - give context to the case study 2. Task - what was the challenge being addressed 3. Approach - What was the architectural approach? 4. Results - What were the results, both measurable and intangible Include the following facts: Facebook’s contributions to NoSQL weren’t limited to caching. They also recognized that **certain new features demanded a different kind of storage system** altogether. A key example was the development of **Facebook’s Messages/Inboxes search** around 2007–2008. The existing MySQL infrastructure was not well-suited for storing and querying the *“reverse indices”* of messages (essentially, each message and its recipients, to enable fast inbox search) under a heavy load. As Facebook’s Avinash Lakshman (one of the authors of Amazon’s Dynamo paper who later joined Facebook) described, *the volume of data, growth rate, and strict latency SLAs for the inbox search feature made it clear that “traditional data storage solutions just wouldn’t fit the bill.”* The team needed something that could **scale writes and data size incrementally in a cost-effective way**, without a single point of failure – similar requirements to what Amazon had faced. Facebook’s solution was to build a new distributed datastore called **Apache Cassandra**. Cassandra, open-sourced by Facebook in 2008, combined ideas from Google’s Bigtable (it uses a column-family data model) and Amazon’s Dynamo (fully distributed, no master nodes, eventual consistency). In Lakshman’s words, *“Hence was born Cassandra… a distributed storage system for managing structured data designed to scale to a very large size across many commodity servers, with no single point of failure”*. Cassandra was engineered for **reliability at massive scale** – it can run on hundreds of nodes across multiple data centers, tolerating constant hardware failures while remaining available. Facebook first deployed Cassandra to power Inbox Search, storing terabytes of index data spread across (at the time) a 600+ core cluster with over 120 TB of disk storage. This architecture met their strict SLA requirements for search latency and throughput. In fact, after the success of inbox search, Facebook had plans to use Cassandra for other similar workloads, though the company later developed other specialized systems too. The important point is that Cassandra **enabled Facebook to add new features that would have been impractical with the existing MySQL setup**. For example, providing fast search over a user’s entire message history (with high write rates as messages are sent) simply wasn’t feasible at Facebook’s scale with normalized SQL tables and synchronous replication. Cassandra’s eventual consistency and schema-flexible design was a better fit, trading off some immediate consistency in favor of availability and write performance – a tradeoff that aligned with Facebook’s needs for user experience. Cassandra proved influential beyond Facebook: once open-sourced, it became one of the leading NoSQL databases adopted by others (Netflix, Reddit, Instagram, and many enterprises). Its design showcasing **tunable consistency, flexible schema, and linear scalability** was a direct result of the lessons from Amazon and Google, validated in Facebook’s environment. In a broader sense, Facebook’s scaling journey taught architects that *“memory is cheap – cache everything”* and that sometimes you must **create new storage engines for new problems**. By 2009, the industry had examples of key-value stores (Dynamo, Voldemort, Riak), document stores (like CouchDB and soon MongoDB), and columnar stores (Bigtable, HBase, Cassandra) – all **NoSQL patterns that were influenced by the pioneering work at companies like Facebook, Amazon, and Google**. It’s notable that Facebook also later developed other NoSQL-like systems, such as **TAO**, a distributed graph cache for the social graph, and **FlockDB**, a simple distributed graph database for friend relationships. These were tailored to specific data patterns (social network feeds, follower graphs) that didn’t map well to normalized SQL tables. Each of these efforts further cemented Facebook’s competitive advantage in delivering new features at scale – they could launch things like the News Feed, real-time messaging, search, etc., without being bottlenecked by their database infrastructure. Finally, create 10 real working references using the following format: ## References 1. [Title](WORKING_URL) - publication_date - publication_name - description_and_relevance

Situation

By 2007-2008, Facebook had reached a critical inflection point in its scaling journey. The social network was experiencing explosive user growth, with hundreds of millions of users generating massive amounts of data daily. While Facebook had successfully addressed some scaling challenges through innovative caching solutions like Memcached, they encountered a fundamental limitation that caching alone could not solve.

Facebook's engineering team, led by key architects including Avinash Lakshman (formerly of Amazon's Dynamo team), recognized that certain new features demanded entirely different storage paradigms. The company's existing MySQL-based infrastructure, while suitable for many traditional web application patterns, was proving inadequate for emerging use cases that required different data access patterns and scaling characteristics.

The most pressing challenge emerged around Facebook's Messages/Inboxes search functionality. This feature required storing and querying complex "reverse indices" of messages—essentially mapping each message to its recipients to enable fast inbox search capabilities. The volume of messaging data, combined with Facebook's strict performance requirements, created a perfect storm that exposed the limitations of traditional relational database approaches at web scale.

As Lakshman later reflected, the scale and performance requirements made it clear that "traditional data storage solutions just wouldn't fit the bill." Facebook needed a solution that could handle not just the current load, but also scale incrementally as the platform continued its rapid growth trajectory.

Task

Facebook's engineering team faced a multi-dimensional challenge that required solving several interconnected problems simultaneously:

Volume and Scale Requirements: The system needed to handle terabytes of index data across hundreds of servers while maintaining consistent performance. The messaging feature alone would generate massive amounts of data as users exchanged billions of messages, each requiring indexing for search functionality.

Write Performance and Latency: Unlike traditional web applications that are primarily read-heavy, the messaging search system required high write throughput as new messages continuously arrived and needed immediate indexing. The system had to meet strict Service Level Agreement (SLA) requirements for both search latency and indexing throughput.

Availability and Fault Tolerance: Facebook's global user base demanded 24/7 availability. The storage system needed to operate without single points of failure and gracefully handle the constant hardware failures inevitable in large-scale distributed systems.

Cost-Effective Scalability: The solution needed to scale incrementally using commodity hardware rather than requiring expensive specialized database servers. This was crucial for Facebook's business model and growth trajectory.

Schema Flexibility: The reverse indexing requirements involved complex, potentially varying data structures that didn't map well to rigid relational schemas. The system needed to accommodate evolving data patterns without requiring expensive schema migrations.

As Lakshman noted, the team needed something that could "scale writes and data size incrementally in a cost-effective way" while maintaining the reliability and performance characteristics required for a feature that would be used by hundreds of millions of users daily.

Approach

Facebook's solution involved creating an entirely new distributed datastore that combined the best architectural ideas from existing systems while addressing their specific requirements. The project, which became Apache Cassandra, represented a synthesis of proven distributed systems concepts adapted for Facebook's unique challenges.

Architectural Foundation: Cassandra's design combined ideas from two influential distributed systems: Google's Bigtable and Amazon's Dynamo. From Bigtable, the team adopted the column-family data model, which provided the flexibility needed for complex indexing structures while maintaining reasonable query performance. From Amazon's Dynamo, they incorporated the fully distributed architecture with no master nodes and eventual consistency guarantees.

Distributed Architecture: The system was designed as a peer-to-peer distributed database where every node in the cluster has the same role—eliminating single points of failure that plagued traditional master-slave architectures. This approach enabled linear scalability and simplified operational management.

Consistency Model: Cassandra implemented tunable consistency, allowing applications to choose the appropriate consistency level for each operation. This flexibility enabled Facebook to optimize for availability and write performance while maintaining sufficient consistency for their use cases.

Data Model: The column-family data model provided the schema flexibility needed for reverse indexing while maintaining the performance characteristics required for high-throughput operations. This model enabled efficient storage and retrieval of the complex data structures required for message search functionality.

Fault Tolerance: The system implemented sophisticated replication and failure detection mechanisms that could tolerate constant hardware failures across large clusters without impacting availability or performance.

As Lakshman described the result: "Hence was born Cassandra… a distributed storage system for managing structured data designed to scale to a very large size across many commodity servers, with no single point of failure."

Results

The deployment and impact of Cassandra at Facebook demonstrated both measurable technical success and broader strategic advantages that influenced the company's competitive position and the industry as a whole.

Measurable Results

Production Deployment: Facebook successfully deployed Cassandra to power their Inbox Search feature, initially running on a cluster of over 600 cores with more than 120 TB of disk storage. The system met Facebook's strict SLA requirements for search latency and throughput, enabling real-time search across users' entire message histories.

Performance Characteristics: The system achieved the write performance and scalability requirements that had been impossible with the existing MySQL infrastructure. Users could search through their complete message history with sub-second response times, even as the platform handled billions of new messages daily.

Operational Efficiency: The distributed architecture eliminated the operational complexity and single points of failure associated with traditional database deployments. The system could automatically handle node failures and traffic redistribution without manual intervention.

Cost Effectiveness: By utilizing commodity hardware in a distributed architecture, Facebook achieved the required performance characteristics at a fraction of the cost of scaling traditional database systems vertically.

Intangible and Strategic Results

Feature Innovation: Cassandra enabled Facebook to implement features that would have been impractical with existing infrastructure. The ability to provide fast search over complete message histories at Facebook's scale represented a competitive advantage that enhanced user engagement and platform stickiness.

Architectural Paradigm Shift: The success of Cassandra validated Facebook's approach of creating specialized storage systems for specific problems rather than trying to adapt general-purpose databases to all use cases. This philosophy influenced subsequent system design decisions across the company.

Industry Influence: Facebook's decision to open-source Cassandra in 2008 created one of the most influential NoSQL databases in the industry. Major companies including Netflix, Reddit, Instagram, and numerous enterprises adopted Cassandra, validating Facebook's architectural decisions and establishing the company as a thought leader in distributed systems.

Ecosystem Development: The success of Cassandra encouraged Facebook to develop other specialized systems including TAO (a distributed graph cache for the social graph) and FlockDB (a distributed graph database for friend relationships). This portfolio of specialized systems provided competitive advantages in launching features like News Feed and real-time messaging.

Knowledge Transfer: The project established valuable precedents and expertise within Facebook's engineering organization. The lessons learned from Cassandra development influenced subsequent projects and established patterns for building large-scale distributed systems.

Strategic Positioning: By 2009, Facebook had demonstrated that it could not only scale existing technologies but also create entirely new systems that the broader industry would adopt. This capability reduced dependence on external vendors and provided technological differentiation that directly supported business objectives.

The broader impact extended beyond Facebook itself. As the industry gained examples of key-value stores (Dynamo, Voldemort, Riak), document stores (CouchDB, MongoDB), and columnar stores (Bigtable, HBase, Cassandra), the NoSQL movement gained momentum. Facebook's contributions, particularly through Cassandra, helped establish the legitimacy of NoSQL approaches and influenced architectural decisions across the technology industry.

The success of Cassandra embodied two key principles that Facebook internalized from their scaling journey: "memory is cheap—cache everything" and the recognition that sometimes organizations must "create new storage engines for new problems." These insights became fundamental to Facebook's engineering culture and competitive strategy.## References

  1. Cassandra – A structured storage system on a P2P Network - August 25, 2008 - Facebook Engineering Blog - Original blog post by Avinash Lakshman announcing Cassandra's development at Facebook for the Inbox Search problem and its open-source release.

  2. Cassandra - A Decentralized Structured Storage System - 2009 - ACM LADIS Workshop - The original academic paper by Avinash Lakshman and Prashant Malik detailing Cassandra's architecture, design decisions, and performance characteristics.

  3. Dynamo: amazon's highly available key-value store - October 2007 - ACM SIGOPS Operating Systems Review - The seminal Amazon Dynamo paper that provided foundational distributed systems concepts adopted by Cassandra.

  4. Bigtable: A Distributed Storage System for Structured Data - June 2008 - ACM Transactions on Computer Systems - Google's influential Bigtable paper that contributed the column-family data model concepts used in Cassandra.

  5. Amazon's Dynamo - October 2, 2007 - All Things Distributed - Werner Vogels' blog post introducing Amazon's Dynamo paper and explaining its significance for distributed systems.

  6. Facebook's Database Handling Billions of Messages - March 11, 2025 - ByteByteGo Newsletter - Recent technical analysis of Cassandra's role in Facebook's messaging infrastructure with performance metrics and architectural insights.

  7. Apache Cassandra - Wikipedia - Ongoing - Wikipedia - Comprehensive overview of Cassandra's development history, technical architecture, and evolution from Facebook's internal project to Apache foundation project.

  8. Dynamo vs Cassandra: Systems Design of NoSQL Databases - October 2, 2018 - Sujith Jay Nair - Detailed technical comparison of Cassandra and Dynamo architectures, highlighting how Cassandra combined concepts from both Dynamo and Bigtable.

  9. The Beauty of Cassandra - April 23, 2018 - Medium - Technical analysis of Cassandra's development by Avinash Lakshman and Prashant Malik, focusing on the Inbox Search problem and architectural decisions.

  10. What is Apache Cassandra? - February 19, 2025 - ScyllaDB - Contemporary technical overview of Cassandra's architecture, development history at Facebook, and current enterprise applications.