Skip to content

Latest commit

 

History

History
89 lines (68 loc) · 3.08 KB

README.md

File metadata and controls

89 lines (68 loc) · 3.08 KB

Testing farm

This document gives a detailed breakdown of the testing processes using testing farm service.

Pre-requisites

  • Python >=3.9
  • TMT command line tool (optional) - for lint and check tmt formatted test plans and tests
    • pip install tmt[all]
  • Testing farm command line tool - for trigger your test plan in testing-farm
    • pip install tft-cli

Links

Current plans and tests

Plans are stored in plans folder, there is a file called main.fmf which contains test plan definition. This definition is composed of hw requirements, prepare steps for created VM executor and specific plans. Specific plan defines selectors for tests which should be executed.

List of plans

  • flink-all
  • smoke
  • flink-sql-example

Usage

Pre-requisites

  1. Get API token for testing farm (how-to obtain token)
  2. Store token into env var export TESTING_FARM_API_TOKEN="your_token"

Run tests

Run all plans

testing-farm request --compose CentOS-Stream-9 --git-url https://github.com/streamshub/flink-sql.git -e IP_FAMILY=ipv4

Select specific plan and git branch

testing-farm request --compose CentOS-Stream-9 \
 --git-url https://github.com/streamshub/flink-sql.git \
 --git-ref some-branch \
 --plan smoke \
 -e IP_FAMILY=ipv4

Run multi-arch build

testing-farm request --compose CentOS-Stream-9 \
 --git-url https://github.com/streamshub/flink-sql.git \
 --git-ref some-branch \
 --plan smoke \
 --arch aarch64,x86_64 \
 -e IP_FAMILY=ipv4

Packit-as-a-service for PR check

Packit-as-a-service is a github application for running testing-farm jobs from PR requested by command. Definition of the jobs is stored in .packit.yaml. Packit can be triggered from the PR by comment, but only members of streamshub organization are able to run tests.

The prepare phase install kind cluster and some required packages it takes around 5min

Usage

Run all jobs for PR

/packit test

Run selected jobs by label

/packit test --labels flink-all

To use different branch or fork of streams-e2e test suite you can override it by env configuration

/packit test --labels smoke --env TEST_REPO_URL=https://github.com/kornys/streams-e2e.git --env TEST_REPO_BRANCH=new-test

List of labels

  • flink-all - all tests with flink tag from streams-e2e, takes around 15min
  • smoke - all tests with flink tag from streams-e2e, takes around 10min
  • flink-sql-example - run just test based on recommendation-app, takes around 7min