Downloading the Internet

Everyone likes to complain when a build tool “downloads the internet” the first time they check out a new project. But the cost is typically amortized over many subsequent builds.

Most build / package tools — Maven, NPM, Apt, even Git sort of — were designed with the assumption that they would be run on a developer’s workstation.

But now we build things on CI servers that start from an empty base image. Every time. They “download the internet” on every run.

Not only does this slow down the CI/CD pipeline, just think of the wasted compute time and bandwidth. Thousands of development shops, millions of builds, terabytes of data shuffled around redundantly.

You can set up caching, for sure, but you have do it, working out the right cache points in the build process. The build tools won’t help you; their assumption of a persistent local filesystem has been sublimated into vapor by The Cloud.