Skip to content

Latest commit

 

History

History
182 lines (123 loc) · 10.4 KB

setting-up-a-new-consumer.md

File metadata and controls

182 lines (123 loc) · 10.4 KB

Setting up a new consumer

To enable a consumer to access our API, they need:

  • a client certificate for mutual TLS with a Subject Distinguished Name(SDN) and Common Name (CN) that matches the authorisation allow list configuration in the application
  • an API key

per environment. This document is split into two parts:

  1. Generate credentials for the consumer
  2. Send the credentials to the consumer

Prerequisites

As a pre-requisite to create a client certificate by running the script below, please ensure that:

  • You are logged in so that you can access AWS resources via the CLI.
  • Verify that you have all your AWS credentials ready by opening the config file in the ".aws" directory.

1. Generate credentials for the consumer

Create a client certificate

Run the generate-client-certificate.sh script with the name of the environment and client.

make generate-client-certificate

This will output three files in the ./scripts/client_certificates directory:

  • a private key e.g. dev-nhs-client.key
  • a certificate signing request (CSR) e.g. dev-nhs-client.csr
  • a client certificate (public key) e.g. dev-nhs-client.pem

The private key must be kept secret and the public key can be shared freely. We share these (along with the API key created below) by encrypting them with a key generated by the consumer.

Create an API key

Within the Cloud Platform Environments GitHub repository and the namespace of the environment:

  1. Create a branch.
  2. Add the client to the clients local in the locals.tf.

This local is used for the API key resource and for the resource to connect the API key to a usage plan, so when clients is updated, these resources will update.

  1. Commit, push and make a pull request.
  2. Wait for all the CI checks to pass.
  3. Check the Terraform plan in Concourse by following the Details link under the concourse-ci/status status check and clicking on the task plan-environments.

The changes should include the additions of a new API key and usage plan key as well as an update to the Kubernetes secret for storing our API keys.

  1. Ask the Cloud Platform team to review the pull request in their Slack channel.
  2. Once the pull request has been approved, merge it.

After some initial CI checks, a comment will be added to the pull request with a link to the Concourse run that will be performing Terraform apply to the namespace. (You'll receive an email when it's been added.)

  1. Follow the Concourse build link and check that the Terraform apply succeeds.

The API key will be automatically generated and saved as a Kubernetes secret for future reference.

You can retrieve this API key with the following command:

kubectl -n hmpps-integration-api-[environment] get secrets consumer-api-keys -o json | jq -r '.data.[client]' | base64 -d
# E.g. kubectl -n hmpps-integration-api-dev get secrets consumer-api-keys -o json | jq -r '.data.bob' | base64 -d

Configure allowed endpoints for the consumer

Add your client common name to the ./src/main/resources/application-[environment].yaml, listing the paths that the new client is allowed to consume. It is important that the name of the client matches the common name exactly.

To view the common name of the client certificate that was just generated, run:

openssl x509 -in ./scripts/client_certificates/[environment]-[consumer]-client.pem -text |grep Subject |grep CN

2. Send the credentials to the consumer

Retrieve the client api key

kubectl -n hmpps-integration-api-<environment> get secrets consumer-api-keys -o json | jq -r '.data.<client>' | base64 -d
# E.g. kubectl -n hmpps-integration-api-dev get secrets consumer-api-keys -o json | jq -r '.data.dev' | base64 -d

Ask the client to generate a key pair and to provide the public key only via email

# Generate private key
openssl genrsa -out hmpps-integration-api-cred-exchange-private-key.pem 3072

# Generate public key
openssl rsa -in hmpps-integration-api-cred-exchange-private-key.pem -pubout -out hmpps-integration-api-cred-exchange-public-key.pem

As size of data we can encrypt with the client's public key is limited we now create a symmetric encryption key and encrypt using the client's public key

# Create a symmetric key
head /dev/urandom | sha256sum > symmetric.key

# Encrypt with client's public key
openssl pkeyutl -encrypt -pubin -inkey hmpps-integration-api-cred-exchange-public-key.pem -in symmetric.key -out symmetric.key.enc

We can now encrypt the client's access credentials for an environment using the symmetric key

# Create a tarball of the access credentials
tar cvfz hmpps-integration-api-preprod.tar.gz preprod/preprod-client.key preprod/preprod-client.pem preprod/preprod-api-key

# Encrypt using symmetric key
openssl enc -aes-256-cbc -pbkdf2 -iter 310000 -md sha256 -salt -in hmpps-integration-api-preprod.tar.gz -out hmpps-integration-api-preprod.tar.gz.enc -pass file:./symmetric.key

Alternatively

If the client's public key is named hmpps-integration-api-cred-exchange-public-key.pem, change directory into the scripts/client_certificates directory and run the zip.sh command. This should complete the above steps.

Accessing the credentials

We can now send the encrypted symmetric key (symmetric.key.enc) and encrypted access credentials (hmpps-integration-api-preprod.tar.gz.enc) to the client via email. The client may now decrypt the symmetric key using their private key and subsequently the access credentials using the symmetric key

# Decrypt symmetric key file with private key
openssl pkeyutl -decrypt -inkey hmpps-integration-api-cred-exchange-private-key.pem -in symmetric.key.enc -out symmetric.key

# Decrypt access credentials using symmetric key
openssl enc -d -aes-256-cbc -pbkdf2 -iter 310000 -md sha256 -salt -in hmpps-integration-api-preprod.tar.gz.enc -out hmpps-integration-api-preprod.tar.gz -pass file:./symmetric.key

Create new consumer subscriber queue for events

Create basic infrastructure

Within the Cloud Platform Environments GitHub repository and the namespace of the environment:

  1. Create a branch.
  2. Add new client subscriber terraform file. Example: event-subscriber-mapps.tf
  3. Rename client name "mapps" to new client name
  4. Add new client filter list secret. example secret.tf
  5. Add a client queue mapping. Example: locals.tf
  6. Follow steps 3-8 in Create an API key to merge branch to main.
  7. Retrieve the client queue name and ARN with the following command: 
    kubectl -n hmpps-integration-api-[environment] get secrets [your queue secret name] -o json
# E.g. kubectl -n hmpps-integration-api-dev get secrets event-mapps-queue  -o json
  8. Send the client queue name and ARN to the consumer

The consumer can use the POST /token endpoint in API Gateway to retrieve temporary credentials, then use the SQS APIs or SDKs to receive and delete messages. For example:

temporary_credentials=$(curl --cert client.pem --key client.key -H "x-api-key: $api_key" -XPOST https://dev.integration-api.hmpps.service.justice.gov.uk/token)
export AWS_ACCESS_KEY_ID=$(jq -r '.AccessKeyId' <<< "$creds")
export AWS_SECRET_ACCESS_KEY=$(jq -r '.SecretAccessKey' <<< "$creds")
export AWS_SESSION_TOKEN=$(jq -r '.SessionToken' <<< "$creds")

aws sqs get-queue-attributes --attribute-names ApproximateNumberOfMessages --queue-url "https://sqs.eu-west-2.amazonaws.com/754256621582/$client_queue_name" --region eu-west-2 --output text
> 1234

Using AWS secret for filter Policy

  1. Login to the AWS Console, navigate to Secrets Manager and navigate to the secret created in the previous step by search using the secret description. e.g. MAPPS event filter list Pre-prod
  2. Click on the secret and then click on Retrieve secret value. If this is your first time accessing the new secret, you will see an error Failed to get the secret value.
  3. Click on Set secret Value, and set the Plaintext value as: {"eventType":["default"]}. Setting filter to default will block subscriber receiving any messages. Event notifier will update the subscriber and AWS secret with actual filter list later.
  4. Save the change
  5. Create new Cloud Platform Environments GitHub repository branch
  6. Update terraform to load the secret value from AWS and update filter_policy value. Follow Example. Note: The name of aws_secretsmanager_secret module has to be same as the secret name created from step 4/5 above.
  7. Follow steps 3-8 in Create an API key to merge branch to main.