Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

workload: enable tenant testing for allccl_test #138915

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

workload: enable tenant testing for allccl_test #138915

wants to merge 1 commit into from
+4 −8

Conversation

shubhamdhama
Copy link
Contributor

Since #75449 is fixed we can enable tenant testing here.

Informs: #138912
Epic: CRDB-38970
Release note: None

@shubhamdhama
workload: enable tenant testing for allccl_test
b5226c3 
Since cockroachdb#75449 is fixed we can enable tenant testing here.

Informs: cockroachdb#138912
Epic: CRDB-38970
Release note: None
@shubhamdhama shubhamdhama requested a review from a team as a code owner January 13, 2025 14:16
@shubhamdhama shubhamdhama requested review from golgeek and DarrylWong and removed request for a team January 13, 2025 14:16
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@shubhamdhama
Copy link
Contributor Author

shubhamdhama commented Jan 13, 2025 •
edited
Loading

Even though the tests works, the time elapsed is quite high for external-process mode,

❯ dev test pkg/ccl/workloadccl/allccl:allccl_test -v --ignore-cache -- --test_env=COCKROACH_TEST_TENANT=false
//pkg/ccl/workloadccl/allccl:allccl_test                                 PASSED in 25.8s
❯ dev test pkg/ccl/workloadccl/allccl:allccl_test -v --ignore-cache -- --test_env=COCKROACH_TEST_TENANT=shared
//pkg/ccl/workloadccl/allccl:allccl_test                                 PASSED in 30.5s
❯ dev test pkg/ccl/workloadccl/allccl:allccl_test -v --ignore-cache -- --test_env=COCKROACH_TEST_TENANT=external
//pkg/ccl/workloadccl/allccl:allccl_test                                 PASSED in 166.0s
detailed time elapsed by each test 
❯ dev test pkg/ccl/workloadccl/allccl:allccl_test -v --ignore-cache -- --test_env=COCKROACH_TEST_TENANT=false
--- PASS: TestConsistentSchema (2.02s)
    --- PASS: TestConsistentSchema/bank (0.00s)
    --- PASS: TestConsistentSchema/bulkingest (0.00s)
    --- PASS: TestConsistentSchema/connectionlatency (0.00s)
    --- PASS: TestConsistentSchema/geospatial (2.00s)
    --- PASS: TestConsistentSchema/indexes (0.00s)
    --- PASS: TestConsistentSchema/insights (0.00s)
    --- PASS: TestConsistentSchema/intro (0.00s)
    --- PASS: TestConsistentSchema/json (0.00s)
    --- PASS: TestConsistentSchema/kv (0.00s)
    --- PASS: TestConsistentSchema/ledger (0.00s)
    --- PASS: TestConsistentSchema/movr (0.00s)
    --- PASS: TestConsistentSchema/querybench (0.00s)
    --- PASS: TestConsistentSchema/querylog (0.00s)
    --- PASS: TestConsistentSchema/queue (0.00s)
    --- PASS: TestConsistentSchema/rand (0.00s)
    --- PASS: TestConsistentSchema/roachmart (0.00s)
    --- PASS: TestConsistentSchema/schemachange (0.00s)
    --- PASS: TestConsistentSchema/sqlsmith (0.00s)
    --- PASS: TestConsistentSchema/sqlstats (0.00s)
    --- PASS: TestConsistentSchema/startrek (0.00s)
    --- PASS: TestConsistentSchema/tpcc (0.00s)
    --- PASS: TestConsistentSchema/tpcc-checks (0.00s)
    --- PASS: TestConsistentSchema/tpccmultidb (0.00s)
    --- PASS: TestConsistentSchema/tpcds (0.00s)
    --- PASS: TestConsistentSchema/tpch (0.00s)
    --- PASS: TestConsistentSchema/ttlbench (0.00s)
    --- PASS: TestConsistentSchema/ttllogger (0.00s)
    --- PASS: TestConsistentSchema/ycsb (0.00s)
--- PASS: TestDeterministicInitialData (4.67s)
    --- PASS: TestDeterministicInitialData/bank (0.00s)
    --- PASS: TestDeterministicInitialData/bulkingest (0.06s)
    --- PASS: TestDeterministicInitialData/indexes (0.00s)
    --- PASS: TestDeterministicInitialData/intro (0.00s)
    --- PASS: TestDeterministicInitialData/json (0.00s)
    --- PASS: TestDeterministicInitialData/ledger (0.19s)
    --- PASS: TestDeterministicInitialData/movr (0.03s)
    --- PASS: TestDeterministicInitialData/queue (0.00s)
    --- PASS: TestDeterministicInitialData/rand (0.00s)
    --- PASS: TestDeterministicInitialData/roachmart (1.44s)
    --- PASS: TestDeterministicInitialData/sqlsmith (0.00s)
    --- PASS: TestDeterministicInitialData/startrek (0.00s)
    --- PASS: TestDeterministicInitialData/tpcc (0.22s)
    --- PASS: TestDeterministicInitialData/tpch (2.71s)
    --- PASS: TestDeterministicInitialData/ycsb (0.02s)
--- PASS: TestAllRegisteredImportFixture (16.28s)
    --- PASS: TestAllRegisteredImportFixture/bank (0.55s)
    --- PASS: TestAllRegisteredImportFixture/bulkingest (0.72s)
    --- PASS: TestAllRegisteredImportFixture/geospatial (6.10s)
    --- PASS: TestAllRegisteredImportFixture/insights (0.43s)
    --- PASS: TestAllRegisteredImportFixture/intro (0.25s)
    --- PASS: TestAllRegisteredImportFixture/ledger (1.71s)
    --- PASS: TestAllRegisteredImportFixture/movr (0.87s)
    --- PASS: TestAllRegisteredImportFixture/tpcc (1.68s)
    --- PASS: TestAllRegisteredImportFixture/tpccmultidb (1.60s)
    --- PASS: TestAllRegisteredImportFixture/ycsb (0.38s)
--- PASS: TestAllRegisteredSetup (24.74s)
    --- PASS: TestAllRegisteredSetup/bank (0.56s)
    --- PASS: TestAllRegisteredSetup/bulkingest (1.57s)
    --- PASS: TestAllRegisteredSetup/connectionlatency (0.29s)
    --- PASS: TestAllRegisteredSetup/geospatial (3.33s)
    --- PASS: TestAllRegisteredSetup/indexes (0.25s)
    --- PASS: TestAllRegisteredSetup/insights (0.43s)
    --- PASS: TestAllRegisteredSetup/intro (0.29s)
    --- PASS: TestAllRegisteredSetup/json (0.28s)
    --- PASS: TestAllRegisteredSetup/kv (0.25s)
    --- PASS: TestAllRegisteredSetup/ledger (1.03s)
    --- PASS: TestAllRegisteredSetup/movr (0.76s)
    --- PASS: TestAllRegisteredSetup/querybench (0.37s)
    --- PASS: TestAllRegisteredSetup/querylog (0.26s)
    --- PASS: TestAllRegisteredSetup/queue (0.26s)
    --- PASS: TestAllRegisteredSetup/rand (0.34s)
    --- PASS: TestAllRegisteredSetup/roachmart (0.22s)
    --- PASS: TestAllRegisteredSetup/schemachange (0.25s)
    --- PASS: TestAllRegisteredSetup/sqlsmith (0.19s)
    --- PASS: TestAllRegisteredSetup/sqlstats (0.23s)
    --- PASS: TestAllRegisteredSetup/startrek (0.39s)
    --- PASS: TestAllRegisteredSetup/tpcc-checks (0.36s)
    --- PASS: TestAllRegisteredSetup/tpccmultidb (11.46s)
    --- PASS: TestAllRegisteredSetup/ttllogger (0.30s)
    --- PASS: TestAllRegisteredSetup/ycsb (0.85s)
//pkg/ccl/workloadccl/allccl:allccl_test                                 PASSED in 25.8s
❯ dev test pkg/ccl/workloadccl/allccl:allccl_test -v --ignore-cache -- --test_env=COCKROACH_TEST_TENANT=shared
--- PASS: TestConsistentSchema (2.07s)
    --- PASS: TestConsistentSchema/bank (0.00s)
    --- PASS: TestConsistentSchema/bulkingest (0.00s)
    --- PASS: TestConsistentSchema/connectionlatency (0.00s)
    --- PASS: TestConsistentSchema/geospatial (2.06s)
    --- PASS: TestConsistentSchema/indexes (0.00s)
    --- PASS: TestConsistentSchema/insights (0.00s)
    --- PASS: TestConsistentSchema/intro (0.00s)
    --- PASS: TestConsistentSchema/json (0.00s)
    --- PASS: TestConsistentSchema/kv (0.00s)
    --- PASS: TestConsistentSchema/ledger (0.00s)
    --- PASS: TestConsistentSchema/movr (0.00s)
    --- PASS: TestConsistentSchema/querybench (0.00s)
    --- PASS: TestConsistentSchema/querylog (0.00s)
    --- PASS: TestConsistentSchema/queue (0.00s)
    --- PASS: TestConsistentSchema/rand (0.00s)
    --- PASS: TestConsistentSchema/roachmart (0.00s)
    --- PASS: TestConsistentSchema/schemachange (0.00s)
    --- PASS: TestConsistentSchema/sqlsmith (0.00s)
    --- PASS: TestConsistentSchema/sqlstats (0.00s)
    --- PASS: TestConsistentSchema/startrek (0.00s)
    --- PASS: TestConsistentSchema/tpcc (0.00s)
    --- PASS: TestConsistentSchema/tpcc-checks (0.00s)
    --- PASS: TestConsistentSchema/tpccmultidb (0.00s)
    --- PASS: TestConsistentSchema/tpcds (0.00s)
    --- PASS: TestConsistentSchema/tpch (0.00s)
    --- PASS: TestConsistentSchema/ttlbench (0.00s)
    --- PASS: TestConsistentSchema/ttllogger (0.00s)
    --- PASS: TestConsistentSchema/ycsb (0.00s)
--- PASS: TestDeterministicInitialData (4.71s)
    --- PASS: TestDeterministicInitialData/bank (0.00s)
    --- PASS: TestDeterministicInitialData/bulkingest (0.05s)
    --- PASS: TestDeterministicInitialData/indexes (0.00s)
    --- PASS: TestDeterministicInitialData/intro (0.00s)
    --- PASS: TestDeterministicInitialData/json (0.00s)
    --- PASS: TestDeterministicInitialData/ledger (0.16s)
    --- PASS: TestDeterministicInitialData/movr (0.09s)
    --- PASS: TestDeterministicInitialData/queue (0.00s)
    --- PASS: TestDeterministicInitialData/rand (0.01s)
    --- PASS: TestDeterministicInitialData/roachmart (1.43s)
    --- PASS: TestDeterministicInitialData/sqlsmith (0.00s)
    --- PASS: TestDeterministicInitialData/startrek (0.00s)
    --- PASS: TestDeterministicInitialData/tpcc (0.22s)
    --- PASS: TestDeterministicInitialData/tpch (2.72s)
    --- PASS: TestDeterministicInitialData/ycsb (0.03s)
--- PASS: TestAllRegisteredImportFixture (16.50s)
    --- PASS: TestAllRegisteredImportFixture/bank (0.78s)
    --- PASS: TestAllRegisteredImportFixture/bulkingest (0.80s)
    --- PASS: TestAllRegisteredImportFixture/geospatial (6.24s)
    --- PASS: TestAllRegisteredImportFixture/insights (0.63s)
    --- PASS: TestAllRegisteredImportFixture/intro (0.39s)
    --- PASS: TestAllRegisteredImportFixture/ledger (0.91s)
    --- PASS: TestAllRegisteredImportFixture/movr (0.92s)
    --- PASS: TestAllRegisteredImportFixture/tpcc (1.76s)
    --- PASS: TestAllRegisteredImportFixture/tpccmultidb (1.71s)
    --- PASS: TestAllRegisteredImportFixture/ycsb (0.51s)
--- PASS: TestAllRegisteredSetup (29.65s)
    --- PASS: TestAllRegisteredSetup/bank (0.75s)
    --- PASS: TestAllRegisteredSetup/bulkingest (1.84s)
    --- PASS: TestAllRegisteredSetup/connectionlatency (0.46s)
    --- PASS: TestAllRegisteredSetup/geospatial (3.62s)
    --- PASS: TestAllRegisteredSetup/indexes (0.46s)
    --- PASS: TestAllRegisteredSetup/insights (0.67s)
    --- PASS: TestAllRegisteredSetup/intro (0.40s)
    --- PASS: TestAllRegisteredSetup/json (0.42s)
    --- PASS: TestAllRegisteredSetup/kv (0.39s)
    --- PASS: TestAllRegisteredSetup/ledger (1.09s)
    --- PASS: TestAllRegisteredSetup/movr (0.83s)
    --- PASS: TestAllRegisteredSetup/querybench (0.43s)
    --- PASS: TestAllRegisteredSetup/querylog (0.48s)
    --- PASS: TestAllRegisteredSetup/queue (0.54s)
    --- PASS: TestAllRegisteredSetup/rand (0.48s)
    --- PASS: TestAllRegisteredSetup/roachmart (0.57s)
    --- PASS: TestAllRegisteredSetup/schemachange (0.52s)
    --- PASS: TestAllRegisteredSetup/sqlsmith (0.46s)
    --- PASS: TestAllRegisteredSetup/sqlstats (0.47s)
    --- PASS: TestAllRegisteredSetup/startrek (0.50s)
    --- PASS: TestAllRegisteredSetup/tpcc-checks (0.46s)
    --- PASS: TestAllRegisteredSetup/tpccmultidb (12.12s)
    --- PASS: TestAllRegisteredSetup/ttllogger (0.41s)
    --- PASS: TestAllRegisteredSetup/ycsb (1.20s)
//pkg/ccl/workloadccl/allccl:allccl_test                                 PASSED in 30.5s
❯ dev test pkg/ccl/workloadccl/allccl:allccl_test -v --ignore-cache -- --test_env=COCKROACH_TEST_TENANT=external
--- PASS: TestConsistentSchema (1.90s)
    --- PASS: TestConsistentSchema/bank (0.00s)
    --- PASS: TestConsistentSchema/bulkingest (0.00s)
    --- PASS: TestConsistentSchema/connectionlatency (0.00s)
    --- PASS: TestConsistentSchema/geospatial (1.89s)
    --- PASS: TestConsistentSchema/indexes (0.00s)
    --- PASS: TestConsistentSchema/insights (0.00s)
    --- PASS: TestConsistentSchema/intro (0.00s)
    --- PASS: TestConsistentSchema/json (0.00s)
    --- PASS: TestConsistentSchema/kv (0.00s)
    --- PASS: TestConsistentSchema/ledger (0.00s)
    --- PASS: TestConsistentSchema/movr (0.00s)
    --- PASS: TestConsistentSchema/querybench (0.00s)
    --- PASS: TestConsistentSchema/querylog (0.00s)
    --- PASS: TestConsistentSchema/queue (0.00s)
    --- PASS: TestConsistentSchema/rand (0.01s)
    --- PASS: TestConsistentSchema/roachmart (0.00s)
    --- PASS: TestConsistentSchema/schemachange (0.00s)
    --- PASS: TestConsistentSchema/sqlsmith (0.00s)
    --- PASS: TestConsistentSchema/sqlstats (0.00s)
    --- PASS: TestConsistentSchema/startrek (0.00s)
    --- PASS: TestConsistentSchema/tpcc (0.00s)
    --- PASS: TestConsistentSchema/tpcc-checks (0.00s)
    --- PASS: TestConsistentSchema/tpccmultidb (0.00s)
    --- PASS: TestConsistentSchema/tpcds (0.00s)
    --- PASS: TestConsistentSchema/tpch (0.00s)
    --- PASS: TestConsistentSchema/ttlbench (0.00s)
    --- PASS: TestConsistentSchema/ttllogger (0.00s)
    --- PASS: TestConsistentSchema/ycsb (0.00s)
--- PASS: TestDeterministicInitialData (4.51s)
    --- PASS: TestDeterministicInitialData/bank (0.00s)
    --- PASS: TestDeterministicInitialData/bulkingest (0.05s)
    --- PASS: TestDeterministicInitialData/indexes (0.00s)
    --- PASS: TestDeterministicInitialData/intro (0.00s)
    --- PASS: TestDeterministicInitialData/json (0.00s)
    --- PASS: TestDeterministicInitialData/ledger (0.19s)
    --- PASS: TestDeterministicInitialData/movr (0.02s)
    --- PASS: TestDeterministicInitialData/queue (0.00s)
    --- PASS: TestDeterministicInitialData/rand (0.00s)
    --- PASS: TestDeterministicInitialData/roachmart (1.40s)
    --- PASS: TestDeterministicInitialData/sqlsmith (0.00s)
    --- PASS: TestDeterministicInitialData/startrek (0.00s)
    --- PASS: TestDeterministicInitialData/tpcc (0.21s)
    --- PASS: TestDeterministicInitialData/tpch (2.62s)
    --- PASS: TestDeterministicInitialData/ycsb (0.02s)
--- PASS: TestAllRegisteredImportFixture (60.96s)
    --- PASS: TestAllRegisteredImportFixture/bank (0.75s)
    --- PASS: TestAllRegisteredImportFixture/bulkingest (9.63s)
    --- PASS: TestAllRegisteredImportFixture/geospatial (11.35s)
    --- PASS: TestAllRegisteredImportFixture/insights (0.67s)
    --- PASS: TestAllRegisteredImportFixture/intro (0.55s)
    --- PASS: TestAllRegisteredImportFixture/ledger (4.50s)
    --- PASS: TestAllRegisteredImportFixture/movr (0.99s)
    --- PASS: TestAllRegisteredImportFixture/tpcc (10.44s)
    --- PASS: TestAllRegisteredImportFixture/tpccmultidb (10.45s)
    --- PASS: TestAllRegisteredImportFixture/ycsb (9.59s)
--- PASS: TestAllRegisteredSetup (165.22s)
    --- PASS: TestAllRegisteredSetup/bank (0.75s)
    --- PASS: TestAllRegisteredSetup/bulkingest (24.16s)
    --- PASS: TestAllRegisteredSetup/connectionlatency (0.51s)
    --- PASS: TestAllRegisteredSetup/geospatial (12.83s)
    --- PASS: TestAllRegisteredSetup/indexes (0.71s)
    --- PASS: TestAllRegisteredSetup/insights (0.66s)
    --- PASS: TestAllRegisteredSetup/intro (0.56s)
    --- PASS: TestAllRegisteredSetup/json (0.48s)
    --- PASS: TestAllRegisteredSetup/kv (0.52s)
    --- PASS: TestAllRegisteredSetup/ledger (13.38s)
    --- PASS: TestAllRegisteredSetup/movr (9.79s)
    --- PASS: TestAllRegisteredSetup/querybench (0.65s)
    --- PASS: TestAllRegisteredSetup/querylog (0.51s)
    --- PASS: TestAllRegisteredSetup/queue (0.70s)
    --- PASS: TestAllRegisteredSetup/rand (0.50s)
    --- PASS: TestAllRegisteredSetup/roachmart (0.51s)
    --- PASS: TestAllRegisteredSetup/schemachange (0.47s)
    --- PASS: TestAllRegisteredSetup/sqlsmith (0.49s)
    --- PASS: TestAllRegisteredSetup/sqlstats (0.49s)
    --- PASS: TestAllRegisteredSetup/startrek (0.51s)
    --- PASS: TestAllRegisteredSetup/tpcc-checks (0.51s)
    --- PASS: TestAllRegisteredSetup/tpccmultidb (75.66s)
    --- PASS: TestAllRegisteredSetup/ttllogger (0.59s)
    --- PASS: TestAllRegisteredSetup/ycsb (19.20s)
//pkg/ccl/workloadccl/allccl:allccl_test                                 PASSED in 166.0s

@cthumuluru-crdb
Copy link
Contributor

Can you please try multiple runs and see if the performance difference is consistent? On my GCE worker they are not consistent. If they are consistent, we can check the profile. Another option is to disable the tests for external tenants and create a tracking ticket to enable them later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@golgeek golgeek Awaiting requested review from golgeek golgeek is a code owner automatically assigned from cockroachdb/test-eng

@DarrylWong DarrylWong Awaiting requested review from DarrylWong DarrylWong is a code owner automatically assigned from cockroachdb/test-eng

@stevendanna stevendanna Awaiting requested review from stevendanna

@cthumuluru-crdb cthumuluru-crdb Awaiting requested review from cthumuluru-crdb

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@shubhamdhama @cockroach-teamcity @cthumuluru-crdb