Translation extraction & synchronisation for client and backend (Open…

…Slides#6571)

.tx/config

@@ -0,0 +1,10 @@
+[main]
+host = https://www.transifex.com
+
+[o:openslides:p:openslides:r:openslides-4x-template-enpot]
+file_filter   = i18n/<lang>.po
+source_file   = i18n/template-en.pot
+type          = PO
+minimum_perc  = 0
+resource_name = OpenSlides 4.x (template-en.pot)
+

DEVELOPMENT.md

@@ -19,7 +19,7 @@ After cloning you need to initialize all submodules:
 Finally, start the development server:
 
     $ make run-dev
-    
+
 (This command won't run without sudo, or without having set up Docker to run without sudo - see their documentation)
 
 You can access the services independently using their corresponding ports
@@ -31,6 +31,59 @@ or access the full stack on
 
 To run all tests of all services, execute `run-service-tests`.
 
+## Translations
+
+Since multiple services (currently: client and backend) make use of translation files, the
+functionality for this is bundled in this repository. The following make commands are available:
+
+-   `make extract-translations`: Extracts all strings which need translations from all services and
+    merges them together into a single `template-en-pot`, which is placed under `i18n/`. You must
+    run `make run-dev` in another terminal before you can execute this command.
+-   `make push-translations`: Push the current template file under `i18n/template-en.pot` to Transifex
+    to allow translating it there.
+-   `make pull-translations`: Pull the translations in all languages available in the client from
+    Transifex and place them in `i18n/`. Copy them into the respective translation folders of all
+    required services.
+
+### Setting up Transifex
+
+The use of the latter two commands requires you to set up the [Transifex
+CLI](https://developers.transifex.com/docs/cli). Following is a short installation manual for
+version 1.6.6, which may or may not work for future versions. If in doubt, consult the Transifex
+docs.
+
+Execute the following command:
+
+```bash
+curl -o- https://raw.githubusercontent.com/transifex/cli/master/install.sh | bash -s -- v1.6.6
+```
+
+To identify to Transifex with the CLI, you have to provide a `.transifexrc` file in your home path:
+
+```
+[https://www.transifex.com]
+rest_hostname = https://rest.api.transifex.com
+token         = API_TOKEN_HERE
+```
+
+Replace `API_TOKEN_HERE` with your API token, which you can generate under
+https://app.transifex.com/user/settings/api/. Now you should be able to execute the commands above.
+
+### Translation workflow
+
+1. After you made some changes, run `make extract-translations` to update the template file.
+1. Run `make push-translations` to push your changes to Transifex.
+1. Translate the new strings on Transifex.
+1. When you are done, run `make pull-translations` to fetch the new translation files.
+1. Create a pull request in all affected repositories.
+
+### Adding new languages
+
+`make pull-translations` only pulls the translation files which are available in the client by
+calling `npm run get-available-translations` inside the client container. If you want to add a new
+language, you must also change the list of available languages in the client to make it available to
+pull via these scripts.
+
 ## Adding a new Service
 
     $ git submodule add <[email protected]>
@@ -76,8 +129,8 @@ The output should be similar to
 
 These environment variables are available:
 
-- `<SERVICE>_HOST`: The host from a required service
-- `<SERVICE>_PORT`: The port from a required service
+-   `<SERVICE>_HOST`: The host from a required service
+-   `<SERVICE>_PORT`: The port from a required service
 
 Required services can be `MESSAGE_BUS`, `DATASTORE_WRITER`, `PERMISSION`, `AUTOUPDATE`,
 etc. For private services (e.g. a database dedicated to exactly one service),
@@ -89,15 +142,15 @@ database user for the datastore: `DATASTORE_DATABASE_USER`.
 A makefile must be provided at the root-level of the service. The currently
 required (phony) targets are:
 
-- `run-tests`: Execute all tests from the submodule
-- `build-dev`: Build an image with the tag `openslides-<service>-dev`
+-   `run-tests`: Execute all tests from the submodule
+-   `build-dev`: Build an image with the tag `openslides-<service>-dev`
 
 ### Build arguments in the Dockerfile
 
 These build arguments should be supported by every service:
 
-- `REPOSITORY_URL`: The git-url for the repository to use
-- `GIT_CHECKOUT`: A branch/tag/commit to check out during the build
+-   `REPOSITORY_URL`: The git-url for the repository to use
+-   `GIT_CHECKOUT`: A branch/tag/commit to check out during the build
 
 Note that meaningful defaults should be provided in the Dockerfile.
 

Makefile

@@ -53,3 +53,19 @@ services-to-main:
 # Will set the upstream remote to "origin"
 submodules-origin-to-upstream:
 	git submodule foreach -q --recursive 'git remote rename origin upstream'
+
+
+# Translation helper targets
+
+extract-translations:
+	dev/scripts/extract-translations.sh
+
+push-translations:
+	tx push --source
+
+pull-translations:
+	tx pull --translations --languages $$(dev/scripts/dc-dev.sh exec client npm run get-available-languages | tail -n 1)
+
+copy-translations:
+	cp i18n/*.po openslides-client/client/src/assets/i18n/
+	cp i18n/*.po openslides-backend/openslides_backend/i18n/messages/

dev/scripts/extract-translations.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+cd "$(dirname $0)"
+
+# extract translations
+./dc-dev.sh exec client npm run extract
+./dc-dev.sh exec backend make extract-translations
+
+# merge translations
+echo "merging POT files..."
+cd ../..
+xgettext --sort-output --no-location --omit-header -o i18n/template-en.pot \
+         openslides-client/client/src/assets/i18n/template-en.pot \
+         openslides-backend/openslides_backend/i18n/messages/template-en.pot