Skip to content

Commit

Permalink
feat: Add tools to cross-compile the Docker image and deploy on a Ras…
Browse files Browse the repository at this point in the history
…pberry Pi (#51)
  • Loading branch information
RemiBardon committed Sep 3, 2024
2 parents b5a27aa + 3353074 commit aedb0dd
Show file tree
Hide file tree
Showing 28 changed files with 1,015 additions and 162 deletions.
1 change: 0 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
.git
target/*
.cargo
ADRs
.vscode
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "scripts/bash-toolbox"]
path = scripts/bash-toolbox
url = https://github.com/RemiBardon/bash-toolbox.git
6 changes: 6 additions & 0 deletions .zed/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@
// For a full list of overridable settings, and general information on folder-specific settings,
// see the documentation: https://zed.dev/docs/configuring-zed#folder-specific-settings
{
"languages": {
"Shell Script": {
"hard_tabs": true,
"format_on_save": "on"
}
},
"language_overrides": {
"JavaScript": {
// Disable language server so we don't get warnings and errors from `redoc.standalone.js`.
Expand Down
68 changes: 41 additions & 27 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,52 +8,59 @@ After release 1.0, contributions will be more than welcome though!

## Tools you need

### `sea-orm-cli`
### `task`

If you work on databse migrations, you will probably need `sea-orm-cli`:
Instead of using [GNU Make], we are using [Task] for its simplicity and flexibility. You can find installation instructions on [taskfile.dev/installation](https://taskfile.dev/installation/), or just run the folowing on macOS:

```bash
cargo install sea-orm-cli
brew install go-task
```

To list all available commands, use:

```bash
task -a
```

### `prosodyctl`
### `cross`

To use `prosodyctl` locally, you need `prosodyctl` and its dependencies.
To build the Docker image locally, we need to cross-compile Rust code for a different architecture. To avoid cluttering your local environment, we use [`cross`] which handles everything transparently. You can find installation instructions on [github.com/cross-rs/cross#installation](https://github.com/cross-rs/cross?tab=readme-ov-file#installation), or just run the folowing:

```bash
brew install prosodyctl
luarocks install luaunbound
cargo install cross --git https://github.com/cross-rs/cross
```

### `sea-orm-cli`

If you work on database migrations, you will probably need `sea-orm-cli`:

```bash
cargo install sea-orm-cli
```

## Updating dependencies

```bash
rustup upgrade && cargo update
make update-redoc
task update
```

## Testing

After you have setup your environment to run smoke tests and integration tests, you can run all of them in a single command using:

```bash
make test
task test
```

### Smoke testing
This has the added benefit of updating the version of Rust used to run the tests in [`README.md`](./README.md).

As explained in [ADR: Write tests with the Gherkin syntax](./ADRs/2024-01-11-a-write-tests-in-gherkin.md),
we are using Gherkin and Cucumber to run tests. Therefore, you can use this command to run the tests:
### Smoke testing

```bash
cargo test --test behavior
task smoke-test
```

You could also run `cargo test` but it runs unit tests in `src/`, which we don't need.

> [!TIP]
> While developing a feature, add a `@testing` tag to a `Feature`, `Rule` or `Scenario` (non-exhaustive)
> and then use `cargo test --test behavior -- --tags '@testing'` to run only matching `Scenario`s.
As explained in [ADR: Write tests with the Gherkin syntax](./ADRs/2024-01-11-a-write-tests-in-gherkin.md), we are using Gherkin and Cucumber to run the tests. Therefore, you won't be able to filter tests using `cargo test`. To do so, add a `@testing` tag to a `Feature`, `Rule` or `Scenario` (non-exhaustive) and then use `task smoke-test -- --tags '@testing'` to run only matching `Scenario`s.

### Integration testing

Expand All @@ -77,17 +84,19 @@ export PROSE_POD_SYSTEM_DIR=???
Finally, since integration tests run on final containers, you have to build `prose-pod-server` and `prose-pod-api`:

```bash
docker build -t proseim/prose-pod-api "${PROSE_POD_API_DIR:?}"
# Build the Prose Pod API image.
task build-image -- --debug
# Build the Prose Pod Server image.
PROSE_POD_SERVER_DIR=???
docker build -t proseim/prose-pod-server "${PROSE_POD_SERVER_DIR:?}"
docker build -t proseim/prose-pod-server:latest "${PROSE_POD_SERVER_DIR:?}"
```

#### Running tests

Then, run the tests using:

```bash
make integration-test
task integration-test
```

If a test fails, Step CI will automatically print some additional information to help you debug the issue. We also print container logs so you can see internal errors.
Expand All @@ -98,16 +107,21 @@ If a test fails, Step CI will automatically print some additional information to
## Building the Docker image

To build the Docker image, you can use the helper script (which builds the image as `proseim/prose-pod-api:latest`):

```bash
docker build -t proseim/prose-pod-api .
task build-image [-- [TARGET_ARCH] [--debug] [--help]]
```
To build the API in debug mode (e.g. to use predictable data generators),
you can use the `CARGO_INSTALL_EXTRA_ARGS` [Docker `ARG`]:
If you don't set `TARGET_ARCH`, `build-image` will build `proseim/prose-pod-api:latest` for your local architecture. If you set `TARGET_ARCH`, `build-image` will build `proseim/prose-pod-api:latest` for the desired architecture. Since the image runs Alpine Linux, which likely doesn't match your local target triple, `build-image` builds the Prose Pod API using [`cross`](#cross). You can set `PROSE_POD_API_IMAGE` to override the final name of the image (useful when cross-compiling).
To build the API in debug mode (e.g. to use predictable data generators), you can use the `--debug` argument:
```bash
docker build -t proseim/prose-pod-api --build-arg CARGO_INSTALL_EXTRA_ARGS='--debug' .
task build-image -- [TARGET_ARCH] --debug
```
[Docker `ARG`]: https://docs.docker.com/reference/dockerfile/#arg "Dockerfile reference | Docker Docs"
[Step CI]: https://stepci.com/ "Step CI homepage"
[Task]: https://stepci.com/ "Task"
[GNU Make]: https://www.gnu.org/software/make/ "Make - GNU Project - Free Software Foundation"
[`cross`]: https://github.com/cross-rs/cross "cross-rs/cross: “Zero setup” cross compilation and “cross testing” of Rust crates"
6 changes: 6 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,12 @@ path = "src/lib.rs"
opt-level = 0
debug = true
debug-assertions = true
strip = "debuginfo"

[profile.staging]
inherits = "dev"
opt-level = 1
lto = "thin"

[profile.release]
opt-level = "s"
Expand Down
23 changes: 5 additions & 18 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,24 +1,11 @@
FROM rustlang/rust:nightly-buster AS build

ARG CARGO_INSTALL_EXTRA_ARGS=''

RUN apt-get update
RUN apt-get install -y musl-tools

RUN rustc --version && \
rustup --version && \
cargo --version
RUN rustup target add x86_64-unknown-linux-musl

WORKDIR /usr/src/prose-pod-api
COPY . .
FROM alpine:latest

RUN cargo install --path . --target x86_64-unknown-linux-musl $CARGO_INSTALL_EXTRA_ARGS
RUN apk update && apk add libgcc libc6-compat

FROM alpine:latest
ARG RUST_OUT_DIR

COPY --from=build /usr/local/cargo/bin/prose-pod-api /usr/local/bin/prose-pod-api
COPY --from=build /usr/src/prose-pod-api/static /static
COPY ${RUST_OUT_DIR:?}/prose-pod-api /usr/local/bin/prose-pod-api
COPY ./static /static

CMD ["prose-pod-api"]

Expand Down
39 changes: 0 additions & 39 deletions Makefile

This file was deleted.

72 changes: 72 additions & 0 deletions Taskfile.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# See <https://taskfile.dev/usage> and <https://taskfile.dev/reference/schema>
# to find all keys and their usage.
version: "3"

tasks:
open-api-docs-ui:
desc: Runs the Prose Pod API and opens the visual API documentation.
cmds:
- ./scripts/open-api-docs-ui
reset-db:
desc: Backs up the database and resets it using `sea-orm-cli`.
cmds:
- ./scripts/reset-db
entity:
desc: Generates database entities using `sea-orm-cli`.
deps: [reset-db]
cmds:
- sea-orm-cli generate entity -o entity/src --lib \
--tables "$(TABLES)" \
--with-serde both \
--serde-skip-deserializing-primary-key \
--serde-skip-hidden-column
# --model-extra-derives 'rocket::form::FromForm'
format-all:
desc: Formats all Rust files.
cmds:
- cargo fmt
format:
desc: Formats modified Rust files.
cmds:
- ./.githooks/pre-commit
test:
desc: Runs all tests and updates "Tested at Rust version" in the `README`.
deps: [smoke-test, integration-test]
cmds:
# NOTE: This is macOS `sed`, out of simplicity since all maintainers use macOS
- 'sed -i '''' ''s/Tested at Rust version: `.+`/Tested at Rust version: `''"$(rustc --version)"''`/g'' README.md'
smoke-test:
desc: Runs smoke tests.
cmds:
- cargo test --test behavior -- {{.CLI_ARGS}}
integration-test:
desc: Runs integration tests.
cmds:
# NOTE: Analytics are temporarily disabled because of [stepci/stepci#239](https://github.com/stepci/stepci/issues/239).
- STEPCI_DISABLE_ANALYTICS=true ./scripts/integration-test {{.CLI_ARGS}}
update:
desc: Updates all dependencies.
deps: [update-redoc]
cmds:
# NOTE: `cargo update` updates all workspace member crates
- echo '[INFO] Updating Rust dependencies…'
- rustup upgrade
- cargo update
# Check for outdated dependencies
- "if cargo install --list | grep -q '^cargo-edit v'; then \
echo '[INFO] Checking for outdated dependencies…'; \
cargo upgrade --dry-run --incompatible --pinned --verbose; \
else \
echo '[WARN] Install `cargo upgrade` with `cargo install cargo-edit` for this script to check for outdated dependencies'; \
fi"
update-redoc:
desc: Updates Redoc.
cmds:
- echo '[INFO] Updating Redoc…'
- wget -q https://cdn.redoc.ly/redoc/latest/bundles/redoc.standalone.js -O static/api-docs/redoc.standalone.js
build-image:
desc: Builds the Prose Pod API Docker image. Run `task build-image -- --help` for more info.
env:
SCRIPTS_ROOT: "{{.ROOT_DIR}}/scripts"
cmds:
- "{{.ROOT_DIR}}/scripts/build-image {{.CLI_ARGS}}"
1 change: 1 addition & 0 deletions scripts/bash-toolbox
Submodule bash-toolbox added at 77c9de
Loading

0 comments on commit aedb0dd

Please sign in to comment.