Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented Docker multi-stage build #1819

Closed
wants to merge 2 commits into from
Closed

Implemented Docker multi-stage build #1819

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5424f72
Implemented Docker multi-stage build
pbratkowski Dec 10, 2024
9fbab4a
Fixed #1817 -- Added tracdb container
pbratkowski Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# pull official base image
FROM python:3.12-slim-bookworm
FROM python:3.12-slim-bookworm AS djangoproject-www-base

# set work directory
WORKDIR /usr/src/app
Expand Down Expand Up @@ -33,13 +33,10 @@ RUN apt-get update \
libc6-dev \
libpq-dev \
zlib1g-dev \
&& python3 -m pip install --no-cache-dir -r ${REQ_FILE} \
&& apt-get purge --assume-yes --auto-remove \
gcc \
libc6-dev \
libpq-dev \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
&& python3 -m pip install --no-cache-dir -r ${REQ_FILE}


FROM djangoproject-www-base AS djangoproject-www-dev

# install node dependencies
COPY ./package.json ./package.json
Expand All @@ -48,5 +45,15 @@ RUN npm install
# copy project
COPY . .


FROM djangoproject-www-dev AS djangoproject-www-prod

RUN apt-get purge --assume-yes --auto-remove \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, the initial layer for djangoproject-www-prod will still have these packages, so this won't actually reduce the size of the prod image. The purpose of removing the packages in the same layer as they are installed as that they don't end up as part of the layer.

The usual way to work around this with a multi-stage build is to start fresh and copy out of the first layer only the needed files, rather than depending on it in full. In my opinion (others will certainly differ), since Python venvs aren't really self-contained (as in this situation, they depend on system libraries), this is often more trouble than its worth for Python images.

A couple alternative ideas:

  • Uninstall the -dev packages only when REQ_FILE is requirements/prod.txt
  • Uninstall the other packages but not libpq-dev always, which should fix the underlying issue

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch, thanks.

I don't think uninstalling other packages would work, as tox installs psycopg from source, and I believe they are all build requirements. At least, purging different packages led to different errors in my tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks. That makes sense. If gcc is needed too, my second suggestion certainly doesn't make sense :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll revisit this tomorrow, but it should be doable via something like a Docker build arg (KEEP_DEPENDENCIES), and making the package uninstall conditional.

gcc \
libc6-dev \
libpq-dev \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*

# ENTRYPOINT is specified only in the local docker-compose.yml to avoid
# accidentally running it in deployed environments.
78 changes: 5 additions & 73 deletions djangoproject/settings/docker.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,76 +1,8 @@
from .common import * # noqa

DATABASES = {
"default": {
"ENGINE": os.environ.get("SQL_ENGINE"),
"NAME": os.environ.get("SQL_DATABASE"),
"USER": os.environ.get("SQL_USER"),
"PASSWORD": os.environ.get("SQL_PASSWORD"),
"HOST": os.environ.get("SQL_HOST"),
"PORT": os.environ.get("SQL_PORT"),
}
}

SECRET_KEY = os.environ.get("SECRET_KEY")

SILENCED_SYSTEM_CHECKS = SILENCED_SYSTEM_CHECKS + [
"django_recaptcha.recaptcha_test_key_error" # Default test keys for development.
]

ALLOWED_HOSTS = [".localhost", "127.0.0.1", "www.127.0.0.1"]

LOCALE_MIDDLEWARE_EXCLUDED_HOSTS = ["docs.djangoproject.localhost"]

DEBUG = True
THUMBNAIL_DEBUG = DEBUG

CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.locmem.LocMemCache",
"LOCATION": "trololololol",
},
"docs-pages": {
"BACKEND": "django.core.cache.backends.locmem.LocMemCache",
"LOCATION": "docs-pages",
},
}

CSRF_COOKIE_SECURE = False

EMAIL_BACKEND = "django.core.mail.backends.console.EmailBackend"

MEDIA_ROOT = str(DATA_DIR.joinpath("media_root"))

SESSION_COOKIE_SECURE = False

STATIC_ROOT = str(DATA_DIR.joinpath("static_root"))

# Docs settings
DOCS_BUILD_ROOT = DATA_DIR.joinpath("djangodocs")
from .dev import * # noqa

# django-hosts settings
if parent_host := SECRETS.get("parent_host"):
PARENT_HOST = parent_host

PARENT_HOST = "localhost:8000"

# django-push settings

PUSH_SSL_CALLBACK = False

# Enable optional components

if DEBUG:
try:
import debug_toolbar # NOQA
except ImportError:
pass
else:
INSTALLED_APPS.append("debug_toolbar")
INTERNAL_IPS = ["127.0.0.1"]
MIDDLEWARE.insert(
MIDDLEWARE.index("django.middleware.common.CommonMiddleware") + 1,
"debug_toolbar.middleware.DebugToolbarMiddleware",
)
MIDDLEWARE.insert(
MIDDLEWARE.index("debug_toolbar.middleware.DebugToolbarMiddleware") + 1,
"djangoproject.middleware.CORSMiddleware",
)
# debug-toolbar settings
INTERNAL_IPS = SECRETS.get("internal_ips", ["127.0.0.1"])
9 changes: 9 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ services:
build:
context: ./
dockerfile: Dockerfile
target: djangoproject-www-dev
args:
- REQ_FILE=requirements/tests.txt
entrypoint: ./docker-entrypoint.dev.sh
Expand All @@ -29,3 +30,11 @@ services:
- POSTGRES_USER=djangoproject
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=djangoproject
tracdb:
image: postgres:14-alpine
environment:
- POSTGRES_USER=code.djangoproject
- POSTGRES_PASSWORD=secret
- POSTGRES_DB=code.djangoproject
volumes:
- ./tracdb/trac.sql:/docker-entrypoint-initdb.d/trac.sql
2 changes: 1 addition & 1 deletion tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ python =

[testenv]
allowlist_externals = make
passenv = DJANGO_SETTINGS_MODULE
passenv = DJANGO_SETTINGS_MODULE, DJANGOPROJECT_DATA_DIR
deps =
tests: -r{toxinidir}/requirements/tests.txt
flake8: flake8
Expand Down
Loading