Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: increase tenant testing coverage #141604

Merged
merged 1 commit into from
Feb 21, 2025

Conversation

shubhamdhama
Copy link
Contributor

@shubhamdhama shubhamdhama commented Feb 18, 2025

This PR continues the work from #140447, replacing occurrences of
createTestServerParams with createTestServerParamsAllowTenants
to enable tenant testing in these tests.

Informs: #140446
Epic: CRDB-38970
Release note: None

@shubhamdhama shubhamdhama requested review from a team as code owners February 18, 2025 05:56
@shubhamdhama shubhamdhama requested review from kev-cao, wenyihu6, golgeek, DarrylWong, kyle-a-wong and mgartner and removed request for a team February 18, 2025 05:56
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the pkg/sql changes lgtm, but some of these tests are owned by the SQL Queries and DR teams, so they should take a look.

Copy link
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2nd commit :lgtm:

Reviewed 15 of 15 files at r1, 16 of 16 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @DarrylWong, @golgeek, @kev-cao, @kyle-a-wong, @mgartner, and @wenyihu6)

(Stacked on cockroachdb#141490.)

This PR continues the work from cockroachdb#140447, replacing occurrences of
`createTestServerParams` with `createTestServerParamsAllowTenants`
to enable tenant testing in these tests.

Informs: cockroachdb#140446
Epic: CRDB-38970
Release note: None
@shubhamdhama shubhamdhama force-pushed the sql-tenant-testing-part-2 branch from bf68b79 to 6f2435c Compare February 21, 2025 12:34
Copy link

⚪ Sysbench [SQL, 3node, oltp_read_write]
Metric Old Commit New Commit Delta Note Threshold
sec/op 9.640m ±2% 9.690m ±3% ~ p=0.684 n=10 3.0%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 10.16k ±1% 10.15k ±0% ~ p=0.470 n=10 2.0%
B/op 2.182Mi ±0% 2.181Mi ±0% ~ p=0.971 n=10 2.0%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/6f2435c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/bin/pkg_sql_tests benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/fbcabe0/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/fbcabe029eb2e1701710d08cebbca6d15089f911/bin/pkg_sql_tests benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/SQL/3node/oltp_read_write$ --old=fbcabe0 --new=6f2435c ./pkg/sql/tests
⚪ Sysbench [KV, 1node, local, oltp_read_only]
Metric Old Commit New Commit Delta Note Threshold
sec/op 672.5µ ±1% 679.5µ ±2% ~ p=0.315 n=10 2.0%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 439.0 ±0% 439.0 ±0% ~ p=1.000 n=10 1.5%
B/op 254.2Ki ±0% 254.2Ki ±0% ~ p=0.210 n=10 1.5%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/6f2435c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/bin/pkg_sql_tests benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/fbcabe0/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/fbcabe029eb2e1701710d08cebbca6d15089f911/bin/pkg_sql_tests benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/1node_local/oltp_read_only$ --old=fbcabe0 --new=6f2435c ./pkg/sql/tests
⚪ Sysbench [KV, 1node, local, oltp_write_only]
Metric Old Commit New Commit Delta Note Threshold
sec/op 1.338m ±1% 1.341m ±1% ~ p=0.684 n=10 2.5%
errs/op 0.000 ±0% 0.000 ±0% ~ p=1.000 n=10 0.0%
allocs/op 1.394k ±0% 1.394k ±0% ~ p=0.469 n=10 1.8%
B/op 290.3Ki ±0% 290.1Ki ±0% ~ p=0.436 n=10 1.8%
Reproduce

benchdiff binaries:

mkdir -p benchdiff/6f2435c/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/bin/pkg_sql_tests benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/6f2435c/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
mkdir -p benchdiff/fbcabe0/bin/1058449141
gcloud storage cp gs://cockroach-microbench-ci/builds/fbcabe029eb2e1701710d08cebbca6d15089f911/bin/pkg_sql_tests benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests
chmod +x benchdiff/fbcabe0/bin/1058449141/cockroachdb_cockroach_pkg_sql_tests

benchdiff command:

benchdiff --run=^BenchmarkSysbench/KV/1node_local/oltp_write_only$ --old=fbcabe0 --new=6f2435c ./pkg/sql/tests
Artifacts

download:

mkdir -p new
gcloud storage cp gs://cockroach-microbench-ci/artifacts/6f2435c712e4ca5c7b72293bfb605a1f70bb998d/13456892060-1/\* new/
mkdir -p old
gcloud storage cp gs://cockroach-microbench-ci/artifacts/fbcabe029eb2e1701710d08cebbca6d15089f911/13456892060-1/\* old/
Legend
  • Neutral: No significant performance change.
  • 🟡 Warning: Slight degradation, likely due to variance, but still within thresholds.
  • 🔴 Regression: Likely performance regression, requiring investigation.
  • 🟢 Improvement: Possible performance gain.

No regressions detected!

built with commit: 6f2435c712e4ca5c7b72293bfb605a1f70bb998d

@shubhamdhama
Copy link
Contributor Author

shubhamdhama commented Feb 21, 2025

After removing the stacked commit, it seems there are no remaining changes that would affect the DR team. So, we should be good to merge. Thanks for the reviews!

bors r =rafiss,yuzefovich

@shubhamdhama
Copy link
Contributor Author

Oops!

bors r=rafiss,yuzefovich

@craig craig bot merged commit 5d1c0f8 into cockroachdb:master Feb 21, 2025
24 checks passed
@shubhamdhama shubhamdhama deleted the sql-tenant-testing-part-2 branch March 1, 2025 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants