Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SC-XXX: Improved speed of docker image building #449

Open
wants to merge 26 commits into
base: master
Choose a base branch
from

Conversation

sprymiker
Copy link
Contributor

@sprymiker sprymiker commented Jun 28, 2023

Description

  • Ticket/Issue: TBC

Inspired by https://github.com/spryker/docker-sdk/tree/onsteroids. Thank you, @oleg-glushak!
Difference:

  • Incorporated as a general building mechanism, not just alternative
  • More efficient dependency tree to increase parallelization and decrease time-to-done
  • Pipeline image contains dev dependency as it is required
  • Parallel pre-pull and post-push are not necessary as docker build bake covers this OOTB
  • External cache sources are not necessary for now as it does not bring any significant time benefits

P.S. Image size concern was not fully considered. Additional activities measurements are needed, including changes in spryker/docker-php as a separate topic.

Related resources

Change log

Improved speed of docker image building

  • Introduced docker bake as the primary build mechanism with a fallback to docker build if bake is not available
  • Adjusted build process by utilizing multistage builds and paralleling builds
  • Optimised cache usage by separating/moving parameters that are changed often (e.g. build hash, stamp)
  • Fixed bug with docker/sdk export when export options were ignored when -v option was used at the same time
  • Added labels for all images containing project revision, project name, and SDK revision
  • Adjusted maintenance mode to be controlled by environment variable: SPRYKER_MAINTENANCE_MODE_ENABLED=1 docker/sdk run
  • Adjusted default progress type to auto for more compatibility

Checklist

  • I agree with the Code Contribution License Agreement in CONTRIBUTING.md

- Introduced `docker bake` as main build mechanism with fallback to `docker build` if bake is not available
- Adjusted build process by utilising multistage builds and paralleling builds
- Optimised cache usage by separating/moving parameters that are changed often (e.g. build hash, stamp)
- Fixed bug with `docker/sdk export` when export options was ignored when `-v` option was used at the same time
- Added labels for all images containing project revision, project name and SDK revision
- Adjusted maintenance mode to be controlled by environment variable: SPRYKER_MAINTENANCE_MODE_ENABLED=1 docker/sdk run
@oleg-glushak
Copy link

Great to see this being worked on @sprymiker.
I'm not entirely sure, if External cache sources are not necessary for now as it does not bring any significant time benefits can be correct. Have we measured how long it takes for sdk export images -d ecr command to complete when running in the Cloud (Codebuild) with these changes?

For comparison: sdk export images-slim -d ecr-slim command using https://github.com/spryker/docker-sdk/tree/onsteroids running in Codebuild takes ~8m30s for spryker/suite-nonsplit

…llowing parallel pushes of application images
- Removed intermediate dump-autoload as it probably takes more time than benifits for the following build
…n 0.7.1

- Upgraded syntax docker version
- Removed parallelisation for fallback build (without buildx)
Optimized pipeline and jenkins images by removing unnecessary binaries (e.g. PGSQL client if database is MYSQL)
@oleg-glushak
Copy link

Any chances for this PR to be merged eventually @zyuzka @alexanderM91

@alexanderM91
Copy link
Collaborator

Any chances for this PR to be merged eventually @zyuzka @alexanderM91

Hi @oleg-glushak,
let's wait for an answer from the product team

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

3 participants