- What is taking most of the time in your builds? Cloning, building, testing?
- Improve cloning times by
- Use a reference repository (under Advanced Clone Behaviors) so that you can point to a local base repo instead of copying the entire history for every project, you may also check out @larsxschneider's cached clone script that also caches submodules and Git LFS objects
- Narrow refspec by selecting "Honor refspec on initial clone" in Advanced Clone Behaviors and specfying an appropriate refspec
- Sparse checkout only the directories that are needed for the operation
- reduced repository sizes by moving large binary files to Git LFS using bfg-repo cleaner
- push based build triggering, + use ngrok or open a port if needed
- get version control system server and artifact repository closer to the build server (maybe by hosting it yourself or moving the build server closer to the rest)
- Improve build times by
- only building components once and transport them to all downstream build jobs and machines
- using an artifact repository for shared components
- SSDs, more cores and more memory
- Optimized compiler switches - mvn client VM, parallel build options, memory options
- use a build system that is caching build results globally and does build parallelization automatically
- Improve test times by
- identifying which tests are still testing used functionality and disable the rest
- rewriting integration and system tests to use more mocking (one process without IPC and file system interaction preferred)
- analyze bottlenecks and silent exceptions using New Relic, Dynatrace, OverOps or just VisualVM
- Break monolythic applications done into multiple components that can be tested in isolation (and only retested if they changed)
- dedicate a person per team to improve test execution time
- test partitioning across multiple machines using Jenkins Parallel Test Executor Plugin (this repository)
- SSDs, more cores and more memory
- learn more about Jenkins Pipelines, its job resuming functionalities and its GitHub Integration
- if you like to use a pre-setup environment using Docker that integrates with GitHub.com, you can follow this tutorial.
- pre-heating of integration test systems (e.g. by unfreezing memory snapshots of already fully up and running application server and database VMs)
- only do smoke tests in build, all tests in nightly build (CI result is now also shown in code browser out of the pull request)
- optimized compiler switches, parallel test runs (Maven 3 now supports this better), compile options like -XX:+TieredCompilation -XX:TieredStopAtLevel=1 for short running mvn invocations
- use different commit status contexts for different tests, only make smoke test status checks mandatory
- find the best tradeoff of number of tests/test duration and code coverage
- use a build system that is automatically caching test results parallelizing tests (if code tested and test have not changed)