Elle: inferring isolation anomalies from experimental observations

In the world of database systems, few things are more daunting for vendors than the prospect of their products being scrutinized by Kyle Kingsbury's Jepsen framework. Jepsen is renowned for its ability to uncover hidden weaknesses in isolation guarantees claimed by databases. The latest addition to the Jepsen family, Elle, takes this a step further by leveraging advanced techniques to detect isolation anomalies in experimental observations. The paper "Elle: Inferring Isolation Anomalies from Experimental Observations" by Kingsbury and Alvaro, presented at VLDB'20, introduces this innovative tool and explores its capabilities.

The abstract of the paper humorously captures the situation: "Experience shows that many databases do not provide the isolation guarantees they claim." Jepsen has long been a powerful tool for capturing execution histories and identifying inconsistencies in these histories. However, traditional methods like Knossos struggle with the NP-complete nature of general linearizability and serializability checks, which become intractable as concurrency increases. This leads to state-space explosion, making it difficult to handle even hundreds of transactions.

Enter Elle, a new checker designed to tackle these challenges. As described in the README, Elle operates like a clever lawyer, identifying sequences of events in a transaction history that could not possibly have occurred in that order. By making these inferences, Elle can prove that the history is inconsistent, thereby uncovering isolation anomalies. The key advantage of Elle is its efficiency: it can process histories of hundreds of thousands of transactions in just tens of seconds. This capability opens up new avenues for stress-testing database systems and pushing the boundaries of what is possible in terms of concurrency and isolation.

The paper evaluates Elle's performance by testing it on a variety of databases, including two SQL databases (TiDB and YugaByteDB), a document database (Fauna), and a graph database (Dgraph). In each case, Elle was able to detect unexpected anomalies, highlighting the need for further refinement of these systems. The details of these findings can be found in Section 6 of the paper, and additional information on the collaboration between Jepsen and the respective vendors to resolve these issues is available on the Jepsen website.

The ultimate goal for a transaction isolation checker is to be versatile and effective. An ideal tool should be able to work with general patterns of transactions, not just specially crafted ones, and should be capable of detecting a wide range of anomalies. Elle represents a significant leap forward in this direction, offering a powerful new way to evaluate the isolation guarantees provided by databases. As the field continues to evolve, tools like Elle will play a crucial role in ensuring that systems meet the high standards of concurrency and consistency that users expect.