Is there any datastore in this series that behaved correctly in a partition? I've seen ElasticSearch, Redis, Riak, Mongo, and all of them crapped their pants.
So far as I can tell, no traditional SQL systems have been tested. I'd lay a small bet that Postgres wouldn't lose data, although I don't even know if it has any kind of masterpool capability.
It's not a very thorough test compared to many of the others. I would guess because PostgreSQL doesn't support multi-master clustering.
However, in the test that was done, PostgreSQL did not lose data. The failure mode was that transactions were committed without the client being notified. Kind of the opposite of data loss - there is data in the database that the client doesn't know about.
I believe you can use prepared transactios and some client smarts to mitigate this depending on your use case . There is some back and forth for sure, and the risk of locking the database is higher.
On Cassandra, some CQL collection operations behaved well (adding elements to a set). Everything else Kyle tested there was demonstrated to lose data.
Kafka lost data all over the place under Jepson; though Kyle offered great respect to the team and expects them to deliver optionally configured safer semantics (at a performance cost) in future releases.
Riak was totally solid when using CRDTs and turning off the insane LWW default.
"As with any experiment, we can only disconfirm hypotheses. This test demonstrates that in the presence of a partition and leader election, Zookeeper is able to maintain the linearizability invariant.
Recommendations
Use Zookeeper. It’s mature, well-designed, and battle-tested."
"Is there no hope? Is there anything we can do to preserve my writes in Riak?
Yes. We can use CRDTs.
If we enable allow-mult in Riak, the vector clock algorithms will present both versions to the client. We can combine those objects together using a merge function.
...
CRDTs preserve 100% of our writes.
...
Moreover, CRDTs are an AP design: we can write safely and consistently even when the cluster is totally partitioned–for example, when no majority exists.
...
Without vector clocks, Cassandra can’t safely change a cell–but writing immutable data is safe. Consequently, Cassandra has evolved around those constraints, allowing you to efficiently journal thousands of cells to a single row, and to retrieve them in sorted order. Instead of modifying a cell, you write each distinct change to its own UUID-keyed cell.
Cassandra’s query language, CQL, provides some collection-oriented data structures around this model: sets, lists, maps, and so forth. They’re CRDTs, though the semantics don’t align with what you’ll find in the INRIA paper–no G-sets, 2P-sets, OR-sets, etc. However, some operations are safe–for instance, adding elements to a CQL set:
All 2000 writes succeeded. :-D
That’s terrific! This is the same behavior we saw with G-sets in Riak. However, not all CQL collection operations are intuitively correct. In particular, I’d be wary of the index-based operations for lists, updating elements in a map, and any type of deletions. Deletes are implemented by writing special tombstone cells, which declare a range of other cells to be ignored. Because Cassandra doesn’t use techniques like OR-sets, you can potentially delete records that haven’t been seen yet–even delete writes from the future. Cassandra users jokingly refer to this behavior as “doomstones”.
The important thing to remember is that because there are no ordering constraints on writes, one’s merge function must still be associative and commutative. Just as we saw with Riak, AP systems require you to reason about order-free data structures. In fact, Cassandra and Riak are (almost) formally equivalent in their consistency semantics–the primary differences are in the granularity of updates, in garbage collection/history compaction, and in performance.
Bottom line: CQL collections are a great idea, and you should use them! Read the specs carefully to figure out whether CQL operations meet your needs, and if they don’t, you can always write your own CRDTs on top of wide rows yourself."
