Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: run ci build related tests in parallel #1907

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 134 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,45 +7,171 @@ on:
workflow_dispatch:

jobs:
# Builds the library and persists it as an artifact.
# Later jobs can then reuse that build artifact, skipping
# the build step.
build:
name: Build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Install dependencies
run: pnpm install

- name: Build
run: pnpm build

- name: Cache build output
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

test_unit:
mattcosta7 marked this conversation as resolved.
Show resolved Hide resolved
name: Unit tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Unit tests
run: pnpm test:unit

- name: Build
run: pnpm build
test_node:
name: Node tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Node.js tests
run: pnpm test:node

test_browser:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth noting that ~85% of the test runtime is occupied by Playwright tests. We have around 200 tests there, and GitHub machines only has 2 cores, which means we are running all of them sequentially. This is the main bottleneck of CI performance for us right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, totally.
In some cases though, being able to re-run the playwright tests a few times would be nice even if that was 7minutes each, without needing to rebuild, and run the other suites first. 3 minutes each x 5 runs is 15 minutes saved wall clock

name: Browser tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Playwright install
run: pnpm exec playwright install --with-deps chromium

- name: Browser tests
run: pnpm test:browser

- name: Native tests
run: pnpm test:native

- name: Upload test artifacts
if: always()
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: test/browser/test-results

test_native:
name: Native tests
needs: build
runs-on: macos-latest
steps:
- name: Checkout
uses: actions/checkout@v3

- uses: pnpm/action-setup@v2
with:
version: 7.12

- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: 'pnpm'

- name: Restore build cache
uses: actions/cache@v3
with:
path: ./lib
key: ${{ runner.os }}-build-${{ github.sha }}
restore-keys: |
${{ runner.os }}-build-

- name: Install dependencies
run: pnpm install

- name: Native tests
run: pnpm test:native
Loading