Skip to content

Docker: Stacks

Jump to bottom
George Brownbridge edited this page Apr 4, 2023 · 16 revisions

This page is out of date. Please follow the instructions for the new stack here (stack-manager) and here (stack-data-uploader).

The stack framework

Stacks are collections of Docker containers which can be configured and run together. In principle, all WA services could be deployed in a single stack, but in practice we allocate services to a stack according to their service type, in order to manage resources better and secure data when deploying on the CMCL servers.

Our approach uses docker-compose to manage the build-time properties of images and run-time properties of containers in each stack. The configuration files (docker-compose.*.yml) can be be found in Deploy/stacks, along with some shell script wrappers that are designed to simplify some common tasks.

When to use the stacks

Our framework has been written with both server deployment and local deployment (on individual laptops and desktops) in mind.

Scenarios in which it makes sense for developers to run stacks locally include:

  • During development
    • if a new service needs to access an existing triplestore or agent, but the version deployed at kg.cmclinnovations.com doesn't yet include all of the necessary data/functionality.
    • if multiple existing services need to be updated at the same time, and have to interact with one another.
  • Just before deployment
    • when a new service is ready to be deployed in a stack and the configuration is being tested locally before changes are pushed to the git repository. (This often involves extra configuration, like directing traffic to the service through our reverse proxy; please contact system administrators at CMCL before attempting this)

Using the wrapper scripts

The shell scripts in Deploy/stacks wrap around docker-compose commands to

  1. build some or all of the images in a stack locally (build-stack.sh),
  2. start some or all of the containers in a stack (start-stack.sh),
  3. stop all of the containers in a stack that's currently running (stop-stack.sh)

In the instructions below, stack_name can be 'agent', 'db' or 'web' and mode can be 'dev' or 'prod'. The latter allows, for example, additional services to be added to the stack on the production server, or for images to include debugging tools on the development server.

Building images

If you make changes to the source files or Dockerfile of an image in one of the stacks, that image can be rebuilt using build-stack.sh. If you want to start a stack without making changes, this step isn't required - see the next section for instructions. Note that images in the stacks should always be built using this wrapper, rather than by running 'docker build ...' or 'docker-compose build ...' separately, as it automatically sets various environment variables and image properties.

To (re)build images, navigate to Deploy/stacks and run

./build-stack.sh [stack_name] [mode] <service_list> <--push>

Optional arguments and flags:

<service_list> : A comma-separated list of the service names (not image tags) that you want to build.
                 If this isn't supplied, all images in the stack are built.
      <--push> : An admin-only option for now, which uploads the new image to our Docker registry.

Example: To build the website and reverse proxy images in 'dev' mode, run:

./build-stack.sh web dev website,reverse_proxy

Points to note:

  • If you're updating an existing image, ensure the correct version is set - see the 'image' node in Deploy/stacks/[stack_name]/docker-compose.build.yml
  • Some services have separate development and production mode images; options are set in docker-compose.build.dev.yml and docker-compose.build.prod.yml and the images are distinguished by including '${MODE}' (dev or prod) in the image tag.
  • Please consult CMCL admins before changing any other properties in docker-compose.build*.yml.

Starting a stack

To spin-up one of the stacks, navigate to Deploy/stacks and run

./start-stack.sh [stack_name] [mode] <--force-pull service_list> <--test> <additional_args>

Optional arguments and flags:

     --force-pull : Pull one or more images from the registry, overwriting them if they exist locally.
                    Used when (e.g.) you need to use an image snapshot that has been updated on the registry
           --test : An admin-only option that shouldn't be required in general use - contact CMCL admins for more details.
<additional_args> : Arguments to pass to 'docker-compose up' when starting the stack.
                  : One use for this is to pass a space-separated list of service names, such that only those services are started and the rest of the stack is ignored.

Example: To start the whole agent stack in 'prod' mode, run

./start-stack.sh agent prod

Points to note:

  • start-stack.sh pulls images from the registry if they don't exist locally.
  • The 'prod' version of the 'web' stack can only be started on CMCL's servers and won't work locally, since it depends on an SSL certificate registered to kg.cmclinnovations.com.

Stopping a stack

To stop a running stack, navigate to Deploy/stacks and run

./stop-stack.sh [stack_name] [mode] <--test>

Optional arguments and flags:

  --test : An admin-only option that shouldn't be required in general use - contact CMCL admins for more details.

Example: To stop the web stack when it's running in 'dev' mode:

  ./stop-stack.sh web dev

Examples

Example 1: Connecting to a local triplestore

Scenario

You're developing an agent that needs access to knowledge graph data, but you'd like to experiment with different data on your own machine before adding it to one of the triplestores at kg.cmclinnovations.com.

Objective

Start a triplestore in the 'db' stack, upload some data to it, then connect your agent to the db stack.

Instructions

  1. Start an instance of the db stack, spinning up a container only for the 'blazegraph-geo' service:
./start-stack.sh db dev blazegraph-geo
  1. Add data to the triplestore (using the browser interface at http://localhost:48083/blazegraph, for example).
  1. Spin-up your agent and connect it to the same Docker network as the db stack. This can be achieved by creating a docker-compose.yml file for your agent and adding the db stack's network ('db-dev') as 'external':
version: "3.8"

services: my_agent: image: my_agent:1.0.0-SNAPSHOT container_name: my_agent # Have the container join both the default network and the db_stack network, so that it can access blazegraph directly networks: - default - db_stack restart: unless-stopped

Set up a default network for this stack and declare 'db-dev' as an external network.

For this to work, you need to have already started the db stack in dev mode with './start-stack.sh db dev ...'

networks: default: name: my_nw db_stack: name: db-dev external: true

Then run 'docker-compose up' in the same directory as the '.yml' file, or right click on the file in VS Code and choose 'Compose Up'.
  1. Connect to the 'blazegraph-geo' container from your agent container. The blazegraph container is accessible via its service name and the port that it exposes (8080), rather than the port mapped on the host (48083), i.e blazegraph-geo:8080.

The exact connection method will depend on what agent needs to do, but you can check that the connection is open by starting a shell inside your agent container with:

docker exec -it my_agent bash

or by right-clicking on the Docker container in VS Code and choosing "Attach Shell". The command below retrieves blazegraph namespaces properties in an RDF file, using curl:

curl http://blazegraph-geo:8080/blazegraph/namespace

Note that this example is easily adapted to connect to other stacks running locally; simply call start-stack.sh [stack_name] [mode], then change the external network name from db-dev to [stack_name]-[mode].

Example 2: Adding a new service to the agent stack

Scenario

You have developed and tested a new agent locally and are ready to add it to the agent stack for server deployment.

Objective

Add settings for your service to the docker-compose*.yml files in Deploy/stacks/agent.

Instructions

As described on the deployment workflow page, before adding your service to one of the main stacks, you should already have tested it in it's own, separate, docker-compose.yml. If you're not sure how to do this, the .yml file shown in Example 1 should be useful as a template. Once this is done, you can transplant your configuration into the agent stack. It's important to note that our framework separates build and deploy configuration options into different .yml files. This makes it significantly easier to control which images in a stack are pulled from the registry and which are built locally.

The following steps describe how to build and deploy your service in the agent stack, then test the result.

  1. Add your build configuration to the 'services' node in Deploy/stacks/agent/docker-compose.build.yml.
services:

[other services]

Set a service name here, avoiding underscores. e.g. "my-agent:"

: # Set an image tag, using an initial version number of "1.0.0-SNAPSHOT" image: docker.cmclinnovations.com/:1.0.0-SNAPSHOT build: # Specify the location of your Dockerfile and source files in the WA repo, relative to Deploy/stacks/agent, e.g. ../../../Agents/MyAgent context: # Set some standard labels that we assign to all of our images labels: authors: <your_email> # The values of ${BUILDER} and ${HASH} are set automatically in the wrapper script. builder: "${BUILDER}" # A brief (one sentence) description of what the agent does description: hash: "${HASH}"

  1. Add your deployment configuration to the 'services' node in Deploy/stacks/agent/docker-compose.deploy.yml.
services:

[other services]

Service name must match the one you used in the build configuration"

: # Image tag must match the one you used in the build configuration, unless you want to deploy an older version. # e.g. you could set the build version to 1.1.0-SNAPSHOT, but keep the deploy version as 1.0.0 image: docker.cmclinnovations.com/:1.0.0-SNAPSHOT # Set a container name. The value of ${CONTAINER_NAME_SUFFIX} is set automatically in the wrapper script. container_name: "${CONTAINER_NAME_SUFFIX}" # The pull policy must be set to this value to avoid overwriting locally-built images pull_policy: if_not_present restart: unless-stopped

  1. Choose a host port for your service. Unless your agent will only be accessed from within the agent stack, you'll need to set one or more host ports so that it can be reached from external locations. We use the following strategy to choose ports 1: 
i) Add 40000 to the port number used by the service inside its container (e.g. 8080 + 40000 = 48080)
ii) Add 1 to the value until you find a port that isn't occupied by one of our services. (i.e. if 48080 to 48085 are occupied, choose 48086)
You can check the occupied ports by running ./Deploy/scripts/list_used_ports.sh
  1. Set the default ('live') host port. Add the following to Deploy/stacks/agent/docker-compose.deploy.live.yml:
services:

[other services]

: ports: - 48086:8080

5. **Set the 'test' port**. To allow the stack to be tested on the servers alongside the live version, we set different ports when running with the '--test' flag.  Add 1000 to the default ('live') port number and set it in Deploy/stacks/agent/docker-compose.deploy.test.yml:

services:

[other services]

: ports: - 49086:8080

6. **Modify the reverse proxy in the web stack to forward requests to your agent.** The configuration will depend on the requirements of your service; contact admins at CMCL before attempting this.
  1. Test that your agent builds and starts correctly via the wrapper scripts. Navigate to Deploy/stacks and run
./build-stack.sh agent dev <service name>
./start-stack.sh agent dev <service name>

If your service starts correctly, you can notify CMCL admins that it is ready to deploy.

Footnotes

[1]: This strategy aims to avoid conflicts with other software and our own services. 50000 would be a more natural choice for the offset, because it shifts values into the dynamic port range, but unfortunately, this overlaps with a range that is reserved on Windows when Docker is installed.

Contents

Clone this wiki locally