CAP theorem explained with one bank balance

The CAP theorem says that when a distributed database loses the network link between its servers, it has to choose. It can stay consistent, so no one ever reads out-of-date data, by refusing some requests. Or it can stay available, answering every request, and accept that some answers will be out of date. It cannot do both until the link comes back.

The three letters in CAP

• Consistency: every read returns the most recent write, or an error.
• Availability: every request to a working server gets an answer, though the answer may be out of date.
• Partition tolerance: the system keeps running when messages between servers are lost or delayed. A split like that is called a partition.

Eric Brewer proposed the idea in a keynote in 2000. Seth Gilbert and Nancy Lynch of MIT proved it in 2002 (Gilbert and Lynch).

Consistency here is not the C in ACID. ACID's consistency is about a transaction leaving the data valid. CAP's is about every server agreeing on the latest value.

One bank balance on two servers

Say a bank stores your balance, $100, on two servers so that either can answer if the other fails. Node A and Node B copy every change to each other.

Now the network link between them breaks. You withdraw $50, and the write lands on Node A. A cannot tell B. A moment later, someone checks your balance, and the request goes to Node B. What should B say?

Choose consistency (CP). Node B knows it can't reach A, so it can't be sure it has the latest balance. It returns an error until the link is back. Nobody sees a wrong number, but some requests fail.

Choose availability (AP). Node B answers with what it has: $100. Every request gets an answer, but this one is out of date. When the link heals, the two servers reconcile.

Neither is wrong. A bank ledger usually wants CP: a failed request is better than a wrong balance. A shopping cart or a social feed usually wants AP: a slightly stale view is better than an error page.

Why "pick two of three" is misleading

CAP is often drawn as a triangle where you pick any two corners. That suggests a database could choose consistency and availability and give up partition tolerance. In practice it can't. Networks do partition: cables get cut, switches fail, cloud zones lose contact. A system that spans more than one machine has to keep running when that happens.

So the real choice is narrower: while a partition lasts, consistency or availability. Brewer made the same point in a 2012 article looking back at the theorem (IEEE Computer).

PACELC: the choice when nothing is broken

Partitions are rare. Most of the time the network works, and CAP says nothing about that time. PACELC, proposed by Daniel Abadi in 2012, covers it: if there is a Partition, choose Availability or Consistency; Else, choose low Latency or Consistency.

The second half matters more day to day. To be consistent, a server has to wait for other replicas to confirm each write, which takes time. To be fast, it answers from the nearest copy, which might be slightly behind.

Real systems sit in different places. etcd and ZooKeeper refuse requests when they can't reach a majority of their servers, so they choose consistency in both cases. Cassandra answers from any replica by default, so it chooses availability and low latency, and lets you ask for stronger consistency per query.

FAQ

Can a database be CA?

Only if it never partitions, which means a single server or a network you trust never to fail. Once data lives on more than one machine across a real network, partitions happen, and a database that claims CA has to pick C or A when they do.

Is MongoDB CP or AP?

It depends on settings. By default MongoDB sends writes and reads to one primary server, and a primary that loses contact with a majority steps down, which is CP behavior. Reading from secondaries trades that for availability.

What is the difference between CAP and ACID?

ACID describes transactions on a database: all or nothing, valid data, no interference, saved for good. CAP describes a database spread across machines, and what it can promise when they lose contact. The word consistency means something different in each.