My first real post will be an ugly collection of quick notes about Gradle summit 2017 Youtube videos.

Order is the same than this link.

My advice is to watch every videos with a nice comment after /!\ indicator, because you don’t need to watch all videos in order to improve your Gradle skills :)

1. Adapting Gradle for the CERN Accelerator Control System - Endre Fejes

Main point: Migrate from Ant to Gradle.

Because Ant is XML based, they used Gradle:

dynamic task generation
POM customisation (like Android productFlavors)
Special dependency handling (configurations inspection)
Usage of init scripts as a way to use XML like build.gradle

10 millions java code, 1000+ projects, 20+ dependency hierarchy

2. Next level performance with Gradle’s Build Cache - Lóránt Pintér

Next step for gradle team after local incremental compilation. Enabled by –build-cache.

Shared build caches:

HTTP build cache
Gradle Enterprise build cache

Between developers: no need to rebuild other people’s changes! Recommended sharing strategy: only CI pushes.

Status:

Java projects with Gradle 4.0 (default in .gradle/caches)
Caching support in Android plugin 3.0 (default in .android/build-cache/)

See Introducing Gradle Build Cache Beta.

3. Pandora’s journey to 10X faster… - Tzvetelin (TK) Gospodinov, Alex Dubrouski

/!\ GOOD IDEA of parent plugin!! But long talk as well…

Tools: Gradle, Nexus, SonarQube

Parent plugin (10"):

eliminate boilerplate from build files
control over dependency versions
best practices, additional metadata

4. Dependencies, distributed code and engineering velocity - Mike McGarr

/!\ GREAT TALK about dependency management and the big-solution used by Netflix!!

Netflix prefers client libraries, means library instead of raw Rest API. So many Netflix internal libs consumed by maven distribution.

How to manage transitive dependencies? Example: A -> B and B -> C, so A -> C (with depends on: ->)

Semantic version is insufficient: which version is the most stable/etc?
Consuming libraries is pretty hard!

Publishers lack organization-wide visibility:

Who is consuming my library?
Who is using this API that I would like to change?

One solution is monorepo but works with gates (slow publishing).

Netflix solution:

Publisher feedback: resolves who depends on me (Astrid tool), and if it’s breaking them by compiling (and tests?) on new version (Niagara tool)
Managed source: reduce consumer pain by automatically update consumers with new versions of lib
Distributed refactoring: on all repo, refactor all API calls and issue pull requests en masse (experimental)

Like a consistent distributed monorepo. And seems like Amazon and Linkedin are taking the same way.

5. Identifying and addressing build performance problems - Stefan Oehme

/!\ GREAT step by step guide to optimize Gradle builds.

Monitor build for all team, allowing to see how much you spend waiting your builds.

Example: 100 projects, 1M LOC, 50K Tests, Gradle 2.14.1

Configuration | Single-line change | Clean build 4s | 25s | 55s

Use latest version:

./gradlew wrapper --gradle-version 4.0

Measure with Gradle Profiler: performance scenario testing.

Red flags:

Startup > 1s. Tip: Use daemon with enough memory.
Configuration time > 1s. Used every time, even during IntelliJ sync.
Single line change > 10s
No op build doing any work at all

Below 10s is probably fine.

If big project, break code into smaller modules!

API / implementation separation
Compile avoidance
Incremental compile See blog post.

Parallelise decoupled tests with maxParallelForks and forkEvery config.

Extract script plugins (apply from “gradle/script.gradle”)
Extract binary plugins: use buildSrc for heavy lifting

See:

6. Multi repo development with Composite Builds - Daz DeBoer

Example presentation about composite builds feature.

Change binary dependencies to a project dependencies.

Benefits now:

cross-repository refactoring
smooth migration from monorepo to multi-repo
faster plugin prototyping
run a continuous, composite build

7. Building great tools for developers at LinkedIn, with… -Szczepan Faber

Like Netflix presentation.

Monolyth:

no versionning between internal components
unified versions of external dependencies

Federation:

Individual versioning of software components
Freedom to choose dependency version

See http://shipkit.org to auto publish for each CI validation.

8. Minutes to seconds maximizing incrementality - Cedric Champeau

Clean is waste of time because:

Incremental build: no need to change outputs if inputs didn’t change.
Incremental task input: no need to change all outputs if only some inputs changed (see IncrementalTaskInputs class for plugin developers).

Compile avoidance (typically compile classpath leakage): cascading recompilation resolved by new api/implementation available with new java-library plugin.

Distinguish compile and runtime classpath allows to care only about ABI changes for compile classpath. Even if version change or jar name changed.

Annotation processors disable incremental compilation except if that’s why we have to use apt and compileOnly keywords.

Variant aware dependency management: no need to generate a jar if we need only to compile, classes directory are fine. Or could be generated later while we’re running our changes.

9. AARAWR! Fantastic bits and where to dex them new - Pierre Yves Ricau

Some hacks around aar format and custom multidex operations. Not useful for common gradle problems.

10. Gradle Android Development State of the Union - Xavier Ducrohet

Changes:

New aapt2 to use new gradle incremental (> gradle 3.4).
Java annotation processor in progress
Since 2.3 incremental zip task.

New ArtifactTransform allowing to consume aar -> exploded-aar -> 14 artifacts like:

android-manifest
android-ress

11. Android Performance Pitfalls - Stefan Oehme

See example project with all best practices.

Most of content is common with his other talk.

Try org.gradle.caching=true especially during git branch changes.

12. AndroidBuilds at Twitter - César Puerta, Michael Evans

/!\ GREAT TALK about how to resolve Gradle issues, in a hacky way (debug, etc)!!

Partial build (-a) allow to build only current module (DANGEROUS)!

Compiler avoidance for java:

java -> java-library plugin
api/implementation configurations
declare annotation processors in processor classpath

org.gradle.caching=true is enabling build-cache (Gradle 4.0). See 8"40 config.

Use yourkit.com Java Profiler. See 15"30.

Use logs! with –info

Trying Android gradle plugin early:

AGP 3.0 enable via flag
define flavor dimensions for API changes
Disable aapt2 See 20" for hack for old/new dependency systems support.

See 24"03 for hack in order to avoid AS/command line separation of builds…

See 25"53 to see how debug gradle:

git clone https://github.com/gradle/gradle.git
./gradlew idea
gr --no-daemon -Dorg.gradle.debug=true task # Wait debugger

See 27"51 to see how use local jar in order to debug AGP.

See 32"22 for Twitter library build example.

See 38" for advices like:

isolate annotation processor
use java module when possible

build-scan is free to use!!

17. Designing and writing Gradle plugins - Benjamin Muschko

/!\ GOOD video for Gradle plugins development.

Tips:

Binary plugin > script plugin for tests, etc.
Prefer statically typed languages: Java, Kotlin or Groovy with @CompileStatic
Prefer public API via “compile gradleApi()”
Read performance guide

See plugins dependencies via “buildEnvironment” task.

Capabilities vs conventions allows to share logic but not constants between plugins (see 17"23). For example BaseAndroidPlugin shared between AndroidApplication and AndroidLibrary.

About implementation:

Use plugin dev plugin: compile ‘java-gradle-plugin’
Prefer writing custom tasks types
modeling DSL-like APIs (23"23)
closure method (25")

18. Distributed Refactoring across Github

Github project

Fix guava issues en masse:

“Netflix rewrite” to define our refactoring rule
Google BigQuery to find Java files
Zeppelin/Spark on Dataproc to run our rule against all sources

Gradle summit 2017 notes