Reducing our monorepo size to improve developer velocity

At Dropbox, the monorepo has been a cornerstone of our development process, enabling seamless collaboration and cross-service development. However, as our products evolved and the repository grew, we faced challenges that threatened our developer velocity. The monorepo, initially a powerful tool, had become a hindrance, with its size reaching 87GB and cloning time exceeding an hour. This not only slowed down our engineering loop but also posed operational risks as we neared GitHub's 100GB repository size limit.

To address these issues, Dropbox embarked on a mission to reduce the monorepo size while maintaining its benefits. The goal was to make the repository more manageable, improve clone times, and ensure long-term sustainability. Through a systematic approach, we managed to shrink the repository from 87GB to just 20GB, a staggering 77% reduction. This transformation not only cut the cloning time to under 15 minutes but also alleviated pressure on our CI infrastructure.

The growth of the monorepo was driven by several factors. Over time, we had accumulated a vast amount of code, including legacy systems, unused libraries, and outdated documentation. Additionally, the integration of new features and services had led to code duplication and bloat. To tackle these issues, we implemented a multi-step strategy that focused on refactoring, archiving, and optimizing the repository structure.

First, we conducted a thorough audit of the monorepo's contents. This involved identifying and categorizing files based on their usage patterns, age, and relevance. We prioritized active projects and services, while flagging those that were obsolete or rarely used. This step helped us pinpoint the sources of bloat and understand which components could be safely removed or archived.

Next, we embarked on a massive cleanup effort. We removed redundant code, outdated documentation, and unnecessary artifacts. This included pruning large binary files, such as old build artifacts and test datasets, which had no place in a version-controlled repository. We also refactored our codebase to eliminate duplication and improve modularity, ensuring that each component served a clear purpose and was easily maintainable.

In addition to these internal optimizations, we explored external solutions to further streamline our workflows. We implemented Git LFS (Large File Storage) to offload large binary assets, such as images and videos, from the main repository. This not only reduced the repository size but also improved the efficiency of our CI pipelines, which no longer had to handle these cumbersome files.

Another critical aspect of our strategy was enhancing our documentation and knowledge management practices. We recognized that the monorepo's growth was partly fueled by inadequate documentation, leading to repeated efforts and confusion among developers. To address this, we invested in creating comprehensive, up-to-date documentation and established a robust knowledge-sharing culture within the organization. This not only reduced the need for redundant code but also empowered developers to work more efficiently.

Throughout this process, we learned valuable lessons about maintaining a large monorepo at scale. The key takeaway was the importance of proactive maintenance and a culture of code quality. Regular audits, strict adherence to best practices, and a commitment to refactoring and optimization are essential for sustaining a healthy monorepo. We also emphasized the need for clear communication and collaboration among teams, as the monorepo is a shared resource that requires collective ownership and stewardship.

In conclusion, reducing the size of our monorepo was a critical step in improving developer velocity and ensuring the long-term sustainability of our development processes. By systematically addressing the root causes of repository bloat and implementing strategic optimizations, we have transformed a potential bottleneck into a high-performing asset. This experience underscores the importance of continuous evaluation and adaptation in managing large-scale code repositories, particularly in dynamic environments like ours. As we continue to innovate and evolve, our monorepo will remain a central component of our infrastructure, but now it is leaner, more efficient, and better positioned to support the growth of Dropbox's products and services.