The case for a learned sorting algorithm

In recent years, machine learning (ML) has become an integral part of modern software development, permeating web giants, large consumer companies, and even traditional enterprises. While ML has made significant strides in various domains, its impact on systems software has been limited. However, recent advancements such as "The case for learned index structures" and SageDB are beginning to change this landscape. Today's focus is on a classic computer science problem: sorting.

The paper "The case for a learned sorting algorithm" by Kristo, Vaidya, et al., presented at SIGMOD’20, explores the potential of machine learning in optimizing sorting algorithms. The authors propose a novel approach called "Learned Sort" that leverages a predictive model to determine the position of each data item in a sorted list. This method aims to outperform traditional sorting algorithms by combining the strengths of machine learning and classical algorithms.

The core idea behind Learned Sort is to train a model on a subset of the data to predict the position of each item in the sorted list. If the model were 100% accurate, it could sort the entire dataset by placing each item in its predicted position. However, achieving 100% accuracy is impractical, as it would require the model to memorize the positions of all items, which is slower than sorting itself. Instead, the authors propose using a model trained on a sample of the data to approximate the cumulative distribution function (CDF) of the dataset. This approximation allows the algorithm to quickly sort the data with minimal overhead.

To evaluate the effectiveness of Learned Sort, the authors tested it on a 1 billion-item dataset. The results were impressive: Learned Sort outperformed the next best competitor, RadixSort, by a factor of 1.49x. Notably, this performance includes the time taken to train the model, demonstrating that the approach is both efficient and practical.

The authors achieved this remarkable result by combining the predictive power of machine learning with the efficiency of classical sorting algorithms. The sorting process involves two main steps. First, the algorithm scans the dataset and places each item in its approximate position based on the model's predictions. This step is fast because it relies on the pre-trained model. Second, the algorithm uses Insertion Sort, a well-suited algorithm for nearly-sorted arrays, to refine the order of the items and produce a fully sorted list.

The key to the success of Learned Sort lies in the efficient training of the predictive model. The authors trained the model on a small subset of the data, which allowed them to capture the underlying distribution of the dataset. This approach ensures that the model generalizes well to the entire dataset, providing accurate predictions for each item's position.

The authors also conducted a thorough analysis of the algorithm's performance, comparing it to several other sorting algorithms, including Merge Sort, Heap Sort, and Quick Sort. Learned Sort consistently outperformed these traditional algorithms, demonstrating its superiority in handling large datasets.

In conclusion, "The case for a learned sorting algorithm" presents a groundbreaking approach to optimizing sorting processes by integrating machine learning with classical algorithms. The authors have shown that by leveraging predictive models to approximate the CDF of a dataset, it is possible to achieve significant speedups over traditional sorting algorithms. This innovation not only addresses a fundamental computer science problem but also opens new avenues for optimizing systems software in general. As machine learning continues to evolve, it will be fascinating to see how these ideas are applied to other areas of computer science and software engineering.