Modernizing our Android build system: Part I, the planning

One of the biggest challenges of the mobile developer community at Dropbox in 2018 was our custom build system. Our build system was slow, hard to use, and didn’t support some use cases which were out of scope of the original design. After 4 months of work by our Mobile Platform team, we were able to remove our unicorn implementation for something much more modern and easy to maintain.

In our new build system, we wanted to improve on a couple of things that our current build system was hindering:

• Make it easy to create new modules
• Allow developers to easily modify the build files
• Improve on build times for local development
• Industry standard approaches and tooling, so an engineer can easily Google their way out of problems
• Low barrier of entry, familiar to new hires
• Integration with Android Studio

At Dropbox, we have a repository for all mobile development, called Xplat. One of the benefits is to easily share source code between our different mobile applications and across platforms. For a while, Dropbox invested heavily in cross-platform development via C++ that worked well for the apps that were developed at that time. We even open-sourced Djinni in 2014 to interface cross-platform C++ library code with platform-specific Java and Objective-C on Android and iOS. Most recently in early 2019, we made the decision to move away from C++ development, read more about why here. However, some of our mission critical libraries will remain on C++ e.g. DocScanner which uses OpenCV.

Since December 2016, Dropbox used a meta-build system to build our two mobile apps: Dropbox and Paper. It was a meta-build system in the sense that, for most of our modules, we didn’t write the Gradle build files by hand. These build files were autogenerated using an in-house system called BMBF. Additionally, BMBF would generate Java source code for our analytics, feature gating, and other common libraries written in C++.

What is BMBF? BMBF ( Buildy McBuildface Basic Modular Build Format) was a tool written in Python to help modularize our mobile code base.

BMBF provided guaranteed layered dependency order and reduced boilerplate in build files.

BMBF used a structured and opinionated file system layout. Then it was able to generate build.gradle and wire in those modules into settings.gradle. This made it ‘easy’ to create new modules or re-use an existing module without sacrificing the benefits that come from using the official tools for each platform.

The following module config file:
device.bmbf.yaml

dependencies:
  - dbx/base/async

java:
  src_maven_dependencies:
    - dagger
    - dagger_compiler
  jvm_test_maven_dependencies:
    - junit

Link to gist

Would get parsed by BMBF’s build system and output a build.gradle file

apply plugin: 'com.android.library'
apply plugin: 'kotlin-kapt'

android {
    sourceSets {
        main {
            manifest.srcFile 'AndroidManifest.xml'
            java.srcDirs = ['java/src']
            test.srcDirs = ['jvm_test/src']
            androidTest.srcDirs = ['android_test/src']
        }
    }
}
dependencies {
    implementation project(':dbx:base:async')
    implementation project(':dbx:base:oxygen')
    implementation commonlibs.dagger2
    kapt annotationprocessors.dagger2_compiler
    api commonlibs.kotlinstdlib
    testImplementation testlibs.junit
}

Link to gist

How did BMBF not meet our needs? BMBF was opinionated, it made it difficult to add functionality to our Gradle scripts that were not built into BMBF. If BMBF didn’t support a workflow that a product engineer required, they would either file a ticket on Mobile Platform, or tried to add the functionality themselves.

BMBF had numerous gotchas and a steep learning curve. BMBF required a very specific file and folder structure. BMBF was not compatible with Gradle incremental builds because build.gradle files were being re-created every time a developer built the app. It was not uncommon for engineers who had been with the company for 6+ months to still had no idea how to create a new module using BMBF. Our Slack channels and help forums were bombarded with questions on how to resolve errors that BMBF presents.

BMBF was initially built to help facilitate modularization and sharing code between iOS and Android. Over the years, the original maintainers that created BMBF moved on to other projects or left the company. Our current engineers were not eager to maintain a legacy meta build system and were more in favor of leveraging a standardized build system. Since we started using BMBF, we stopped writing cross-platform modules and even wanted to rewrite C++ modules into platform-specific iOS and Android code. The time had come to revamp our build system.

Our team worked for several weeks on evaluating and analyzing our options to review:

1. Gradle + BMBF (status quo)
2. Gradle only
3. Bazel
4. Buck

Our team created a sandbox environment to accurately compare different build systems.

Gradle Only
No work was needed for Gradle, as it was our baseline.

Bazel
While Bazel was widely used at Dropbox, its use didn’t propagate to our mobile teams. An engineer from Developer Infrastructure worked on generating all the BUILD.bzl files required for all our modules in order to successfully build an APK.

Buck
We did not evaluate Buck because it was not well supported by the community at the time and did not support Kotlin. Also, we did not have any in-house expertise in Buck at the time. We would have needed to dedicate 1-2 mobile engineers full time to work on Buck.

Using the prototypes, we compared the three different build systems’ build times and developer experience.

Build times	Bazel Only	Gradle + BMBF Build	Gradle only
Clean Build no cache	638 s	252 s (with buck cache)	~ 258 seconds
Clean Build w/cache	81.459s	N/A	N/A
NoOp Build	1.334s	20-30 seconds	~ 28.6 seconds
[Incremental] Main module modified	~36 sec	~181 seconds	~ 124 seconds
[Incremental] Shared library module modified	~29 sec	~200 seconds	~ 108 seconds

Bazel Build System:

• Decision: Invest heavily in Bazel now and move C++ and Java/Kotlin development to Bazel.
• Options: This will require upfront investment in a Bazel MVP, as well as continued support to make up for new/missing features that are only released in Gradle by Google. As of December 2018, Google is still working on open sourcing some missing critical pieces of Bazel Android from the internal version, Blaze. For example, App Bundles, which was released on May 2018, is still not available on Bazel as of Dec 2018.
• Pros ▲
  • Tool chain managed by Dropbox Developer Platform team
  • Bazel is widely used in Dropbox
  • Tool built and maintained externally
  • Unified build system for C++ and Java/Kotlin
  • Scales to larger code bases in terms of performance
• Cons ▼
  • Not an industry standard for mobile
  • Latest and greatest features and libraries are not available
  • Requires a bigger upfront investment than migrating to Gradle
  • Requires continuous long term support from Dropbox Developer Platform team until Bazel becomes mobile industry standard (Currently we know only of Google as a user of Blaze for Android)
  • The Android Studio team at Google is focusing on Gradle, at the expense of Bazel
  • The Bazel team at Google is working on adding support for Android and open to feedback but will always be trailing Gradle by 1-2 quarters

Gradle Build System:

• Decision: Make a smaller, strategic, investment to move Java/Kotlin development to Gradle by checking in the project files and removing BMBF from Gradle model management.
• Options: Code generation and C++ development will continue to be done by BMBF. Invest in some guardrails (lint, Herald, templating) to make working with Gradle easy for developers.
• Pros ▲
  • Industry standard for mobile
  • Latest and greatest features and libraries are available
  • Tool built and maintained externally
  • Low migration cost
  • Low maintenance cost
• Cons ▼
  • Poor support for cross platform C++ development. Building xplat C++ code is delegated to BUCK via BMBF
  • Will require some support from Mobile platform over time (e.g version bumps, guardrails, maintenance)

BMBF Build System

• Decision: Keep and maintain BMBF as the mobile build tool
• Pros ▲
  • Good support for cross platform C++ code
  • Harder to break Gradle configurations through developer error
  • New features can be made available by prioritizing work on them in-house
• Cons ▼
  • (Unenthusiastically) managed and maintained by our team, rather than a separate build team
  • Tool is not built and maintained externally
  • Not an industry standard for mobile
  • Will require continuous long term support. New features will require in-house investment

Why not Buck as a build tool

• Decision: Buck is an optional build tool however we didn’t evaluate it as a serious option do to its clear lack of community support and drawbacks.
• Options: Buck is designed to address massively modularized code bases (100+ modules). It also would require significant expertise to maintain and support. As an example Uber has a 3 person team dedicated to Buck support, one of whom is the author of OkBuck which allows using Buck with Gradle projects.
• Pros ▲
  • Good support for cross platform C++ code and caching
• Cons ▼
  • Managed by mobile engineers
  • Not well supported by the community and Facebook (at least for external users)
  • Not an industry standard for mobile

Full Trade-offs Table

We decided to move forward last year with the Gradle Build System, and we will soon be revisiting Bazel . The cheap migration cost from BMBF to the underlying Gradle lead to the decision to first deprecate BMBF.

Although Bazel was extremely fast with regard to the the build time, we were concerned about deteriorating the local developer experience. At the time of our evaluation, Bazel was not very mature. Gradle is the industry standard for building Android apps. Tooling and libraries available for Gradle will take time for it to become available for Bazel.

Find out how we implemented our new Android build system in part II of this post. If you are an Android or iOS engineer who gets excited about solving problems at scale and sharing your findings with the community we’d love for you to come join the team!