Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multi-scope configuration settings #10300

Draft
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

abh1sar
Copy link
Collaborator

@abh1sar abh1sar commented Jan 29, 2025

Description

This PR introduces the concept of multi-scope configuration settings. In addition to the Global level, currently all configurations can be set at a single scope level.
It will be useful if a configuration can be set at multiple scopes. For example, a configuration set at the domain level
will apply for all accounts, but it can be set for an account as well. In which case the account level setting will override the domain level setting.

This is done by changing the column scope of table configuration from string (single scope) to bitmask (multiple scopes).

public enum Scope {
    Global(null, 1),
    Zone(Global, 1 << 1),
    Cluster(Zone, 1 << 2),
    StoragePool(Cluster, 1 << 3),
    ManagementServer(Global, 1 << 4),
    ImageStore(Zone, 1 << 5),
    Domain(Global, 1 << 6),
    Account(Domain, 1 << 7);

Each scope is also assigned a parent scope. When a configuration for a given scope is not defined but is available for multiple scope types, the value will be retrieved from the parent scope. If there is no parent scope or if the configuration is defined for a single scope only, the value will fall back to the global level.

Hierarchy for different scopes is defined as below :

  • Global
    • Zone
      • Cluster
        • Storage Pool
      • Image Store
    • Management Server
    • Domain
      • Account

This PR also updates the scope of the following configurations (Storage Pool scope is added in addition to the existing Zone scope):

  • pool.storage.allocated.capacity.disablethreshold
  • pool.storage.allocated.resize.capacity.disablethreshold
  • pool.storage.capacity.disablethreshold

Doc PR : apache/cloudstack-documentation#476

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

Tested Upgrade path
Tested that the configuration values are set correctly for all mentioned configurations at different scopes.

How did you try to break this feature and the system with this change?

Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 52.92096% with 137 lines in your changes missing coverage. Please review.

Project coverage is 16.07%. Comparing base (cd81398) to head (e20bc6e).
Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
...ava/com/cloud/upgrade/dao/Upgrade42010to42100.java 0.00% 24 Missing ⚠️
.../apache/cloudstack/framework/config/ConfigKey.java 78.70% 11 Missing and 12 partials ⚠️
...udstack/framework/config/impl/ConfigDepotImpl.java 20.00% 19 Missing and 1 partial ⚠️
...va/com/cloud/upgrade/dao/DatabaseAccessObject.java 65.38% 7 Missing and 2 partials ⚠️
...m/cloud/storage/dao/StoragePoolDetailsDaoImpl.java 0.00% 8 Missing ⚠️
.../main/java/com/cloud/dc/ClusterDetailsDaoImpl.java 0.00% 6 Missing ⚠️
...ain/java/com/cloud/user/AccountDetailsDaoImpl.java 0.00% 6 Missing ⚠️
...storage/datastore/db/ImageStoreDetailsDaoImpl.java 0.00% 6 Missing ⚠️
...ain/java/com/cloud/upgrade/dao/DbUpgradeUtils.java 54.54% 3 Missing and 2 partials ⚠️
.../cloud/configuration/ConfigurationManagerImpl.java 54.54% 2 Missing and 3 partials ⚠️
... and 10 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #10300      +/-   ##
============================================
+ Coverage     16.02%   16.07%   +0.05%     
- Complexity    13147    13180      +33     
============================================
  Files          5658     5658              
  Lines        496312   496523     +211     
  Branches      60109    60137      +28     
============================================
+ Hits          79527    79829     +302     
+ Misses       407943   407799     -144     
- Partials       8842     8895      +53     
Flag Coverage Δ
uitests 4.01% <ø> (ø)
unittests 16.92% <52.92%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@abh1sar abh1sar changed the title [WIP] Support multiple hierarchical scopes for configurations [WIP] Add allocation and disable thresholds per storage pool Jan 30, 2025
@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 30, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 12277

@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 30, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12278

@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 31, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12282

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12301

@abh1sar
Copy link
Collaborator Author

abh1sar commented Feb 3, 2025

@blueorangutan test

@blueorangutan
Copy link

@abh1sar a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-12265)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 62469 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12265-kvm-ol8.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_11_isolated_network_with_dynamic_routed_mode Error 2.26 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.37 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.37 test_ipv4_routing.py
test_12_start_vm_multiple_volumes_allocated Error 20.07 test_vm_life_cycle.py

Copy link
Contributor

@shwstppr shwstppr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks goog in general. Some comments added @abh1sar


public static void changeTableColumnIfNotExist(Connection conn, String tableName, String oldColumnName, String newColumnName, String columnDefinition) {
if (dao.columnExists(conn, tableName, oldColumnName)) {
System.out.println("column exists------------------------" + oldColumnName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be removed?


protected void migrateConfigurationScopeToBitmask(Connection conn) {
String scopeDataType = DbUpgradeUtils.getTableColumnType(conn, "configuration", "scope");
logger.info("------------------------{}", scopeDataType);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be removed or rephrased if needed

@@ -183,10 +185,15 @@ public void setDescription(String description) {
}

@Override
public String getScope() {
public int getScope() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abh1sar we have defined scope as Integer but returning int. Please check if this won't cause an issue

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not a problem as long as scope is not null which can't happen as the column is declared as not null.

@abh1sar abh1sar requested a review from shwstppr February 4, 2025 06:50
@abh1sar
Copy link
Collaborator Author

abh1sar commented Feb 4, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12321

@harikrishna-patnala
Copy link
Contributor

@blueorangutan test

@blueorangutan
Copy link

@harikrishna-patnala a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@abh1sar abh1sar changed the title Add allocation and disable thresholds per storage pool Support multi-scope configuration settings Feb 5, 2025
@blueorangutan
Copy link

[SF] Trillian test result (tid-12315)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 58031 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12315-kvm-ol8.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_11_isolated_network_with_dynamic_routed_mode Error 2.34 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.47 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.47 test_ipv4_routing.py
test_12_start_vm_multiple_volumes_allocated Error 13.98 test_vm_life_cycle.py

Copy link
Contributor

@harikrishna-patnala harikrishna-patnala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code LGTM

Copy link
Contributor

@shwstppr shwstppr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM

@borisstoyanov
Copy link
Contributor

@blueorangutan package

@blueorangutan
Copy link

@borisstoyanov a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12399

@borisstoyanov
Copy link
Contributor

@blueorangutan test matrix

@blueorangutan
Copy link

@borisstoyanov a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests

@blueorangutan
Copy link

[SF] Trillian test result (tid-12370)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57252 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12370-kvm-ol8.zip
Smoke tests completed. 139 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_11_isolated_network_with_dynamic_routed_mode Error 2.26 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 2.35 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 2.36 test_ipv4_routing.py
test_12_start_vm_multiple_volumes_allocated Error 14.86 test_vm_life_cycle.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-12372)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 68883 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12372-vmware-70u3.zip
Smoke tests completed. 135 look OK, 6 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_11_isolated_network_with_dynamic_routed_mode Error 2.27 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.38 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.39 test_ipv4_routing.py
test_deploy_more_vms_than_limit_allows Error 170.78 test_deploy_vms_in_parallel.py
test_01_prepare_and_cancel_maintenance Error 0.11 test_ms_maintenance_and_safe_shutdown.py
test_01_deployVMInSharedNetwork Failure 1807.94 test_network.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:teardown Error 1808.10 test_network.py
test_02_restore_vm_with_disk_offering Error 62.21 test_restore_vm.py
test_03_restore_vm_with_disk_offering_custom_size Error 60.13 test_restore_vm.py
test_12_start_vm_multiple_volumes_allocated Error 17.01 test_vm_life_cycle.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-12371)
Environment: kvm-ubuntu22 (x2), Advanced Networking with Mgmt server u22
Total time taken: 73763 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12371-kvm-ubuntu22.zip
Smoke tests completed. 136 look OK, 5 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_11_isolated_network_with_dynamic_routed_mode Error 2.29 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.46 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 3.46 test_ipv4_routing.py
test_oobm_multiple_mgmt_server_ownership Failure 30.75 test_outofbandmanagement.py
test_02_list_cpvm_vm Failure 0.04 test_ssvm.py
test_04_cpvm_internals Failure 0.04 test_ssvm.py
test_12_start_vm_multiple_volumes_allocated Error 17.04 test_vm_life_cycle.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Error 4357.82 test_vpc_redundant.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Error 4358.00 test_vpc_redundant.py
test_02_redundant_VPC_default_routes Error 3615.20 test_vpc_redundant.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-12373)
Environment: xcpng82 (x2), Advanced Networking with Mgmt server ol9
Total time taken: 78384 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10300-t12373-xcpng82.zip
Smoke tests completed. 133 look OK, 8 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
test_01_condensed_drs_algorithm Failure 193.66 test_cluster_drs.py
test_02_balanced_drs_algorithm Failure 186.36 test_cluster_drs.py
test_11_isolated_network_with_dynamic_routed_mode Error 2.27 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 4.38 test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode Error 4.38 test_ipv4_routing.py
test_01_prepare_and_cancel_maintenance Error 0.11 test_ms_maintenance_and_safe_shutdown.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup Error 10.49 test_network.py
test_01_non_strict_host_anti_affinity Error 268.78 test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity Error 122.26 test_nonstrict_affinity_group.py
test_02_create_volume Error 2.19 test_resource_names.py
test_05_scale_vm_dont_allow_disk_offering_change Failure 70.31 test_scale_vm.py
test_12_start_vm_multiple_volumes_allocated Error 14.84 test_vm_life_cycle.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants