-
Notifications
You must be signed in to change notification settings - Fork 217
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(capture): add managed storage account support (#575)
# Description This PR introduces a managed storage account solution to Retina Capture, which managed the storage account resources on behalf the user under the azure resource group specified in the azure credential config file. After this, the user does not need to create even the secret and the captured network artificats will be uploaded to the storage account. ## Related Issue If this pull request is related to any issue, please mention it here. Additionally, make sure that the issue is assigned to you before submitting this pull request. ## Checklist - [x] I have read the [contributing documentation](https://retina.sh/docs/contributing). - [x] I signed and signed-off the commits (`git commit -S -s ...`). See [this documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) on signing commits. - [x] I have correctly attributed the author(s) of the code. - [x] I have tested the changes locally. - [x] I have followed the project's style guidelines. - [x] I have updated the documentation, if necessary. - [ ] I have added tests, if applicable. ## Screenshots (if applicable) or Testing Completed - After updating the helm charts - a storage account and a management policy rule to auto-delete blob after 7 days are created  - After applying a Capture - a blob container with retention policy is created  - a k8s secret is created  - network artifacts are uploaded the container after Capture duration  - After deleting the capture - the secret is deleted  ## Additional Notes Add any additional notes or context about the pull request here. --- Please refer to the [CONTRIBUTING.md](../CONTRIBUTING.md) file for more information on how to contribute to this project. --------- Signed-off-by: Qingchuan Hao <[email protected]>
- Loading branch information
Showing
17 changed files
with
816 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| # Managed Storage Account | ||
|
|
||
| ## Motivation | ||
|
|
||
| Retina Capture helps customers capture network packets on Kuberentes cluster to debug network issues. Before Retina Capture can debug the packets, Retina Capture can store the network packets and the customers need to download the packets from Retina Capture supported locations with tools like Wireshark. | ||
|
|
||
| To simplify customers' work to decide where to store the packets and then download the packets to local environment, we propose a managed account solution for customer to help customer manage the lifecycle, security when to the storage account and garbage collection of the storage account. Customers can use managed storage account by default, or specify their own storage account ID and Storage Blob Data Owner or higher privilege should be granted to [AKS managed identity or service principal](https://learn.microsoft.com/en-us/azure/aks/use-managed-identity). | ||
|
|
||
| ## Detailed Design | ||
|
|
||
| ### Workflow | ||
|
|
||
|  | ||
|
|
||
| ## Setup | ||
|
|
||
| To enable the managed storage account, you need to specify the following configuration in the helm command, | ||
|
|
||
| ```bash | ||
| --set operator.enabled=true \ | ||
| --set operator.capture.enableManagedStorageAccount=true | ||
| ``` | ||
|
|
||
| Internally, `enableManagedStorageAccount` will change the following retina-operator configuration and azure credential config as explained in the following two sections. | ||
|
|
||
| ### Retina-Operator Configuration | ||
|
|
||
| #### enableManagedStorageAccount Configuration | ||
|
|
||
| `enableManagedStorageAccount` configuration controls whether to use managed storage account or not. | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: retina-operator-config | ||
| namespace: kube-system | ||
| data: | ||
| ... ... | ||
| enableManagedStorageAccount: true/false | ||
| azureCredentialConfig: /etc/kubernetes/cloud-config/azure.json | ||
| ``` | ||
| #### Azure credential configuration | ||
| The configuration `azureCredentialConfig` in retina-operator-config indicates the Azure credential file path to talk to azure for the managed resources. | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: retina-operator-config | ||
| namespace: kube-system | ||
| data: | ||
| .. ... | ||
| enableManagedStorageAccount: true/false | ||
| azureCredentialConfig: /etc/kubernetes/cloud-config/azure.json | ||
| ``` | ||
|
|
||
| In the above configuration, when `enableManagedStorageAccount` is true, retina-operator will pick azure credential configuration from `/etc/kubernetes/cloud-config/azure.json`. [retina-operator helm template](https://github.com/microsoft/retina/blob/main/deploy/legacy/manifests/controller/helm/retina/templates/operator.yaml) mounts the secret containing azure credentials to `/etc/kubernetes/cloud-config/azure.json`. | ||
|
|
||
| When default storage account is enabled, a managed storage account will be created under the resource group of the sub both specified in the credential file. | ||
| In the case of AKS, the resource group will be MC, or node, resource group, and subscription will be the overlay subscription. | ||
|
|
||
| #### Azure credential configuration content | ||
|
|
||
| Detailed explanation of the auth configuration can be found [here](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/0556803/pkg/azclient/auth_conf.go). | ||
|
|
||
| Under the resource group, the service principal or managed identity should have `Storage Blob Data Contributor` role to create user delegation SAS, AND at least `Storage Account Contributor` role to manage storage account, container and polices in Azure. | ||
|
|
||
| - Managed Identity | ||
|
|
||
| ```json | ||
| { | ||
| "cloud": "AzurePublicCloud", | ||
| "tenantId": "<tenant-id>", | ||
| "subscriptionId": "<subscription-id>", | ||
| "aadClientId": "msi", | ||
| "aadClientSecret": "msi", | ||
| "resourceGroup": "<resource-group-name>", // the resource group where the cluster is deployed in | ||
| "location": "<location>", // the location where the cluster is deployed in | ||
| "useManagedIdentityExtension": true, | ||
| "userAssignedIdentityID": "<managed-identity-client-id>", | ||
| "useInstanceMetadata": true | ||
| } | ||
| ``` | ||
|
|
||
| - Service Principal | ||
|
|
||
| ```json | ||
| { | ||
| "cloud": "AzurePublicCloud", | ||
| "tenantId": "<tenant-id>", | ||
| "subscriptionId": "<subscription-id>", | ||
| "aadClientId": "<client-id>", | ||
| "aadClientSecret": "<client-secret>", | ||
| "resourceGroup": "<resource-group-name>", // the resource group where the cluster is deployed in | ||
| "location": "<location>", // the location where the cluster is deployed in | ||
| "useManagedIdentityExtension": false, | ||
| "userAssignedIdentityID": "", | ||
| "useInstanceMetadata": true | ||
| } | ||
| ``` | ||
|
|
||
| ### Storage Account | ||
|
|
||
| The storage account, `retina-uuid`, is created under MC resource group and lifecycle management with 7 expiration days is created by Retina Operator if Capture is enabled. AKS customers can enable Retina Capture by upgrading their clusters after this feature is released. | ||
|
|
||
| ### Container | ||
|
|
||
| A container, `capture`, will be created to store the network artifacts after the storage account is ready. And to protect the packets from being deleted or modified, a [retention policy](https://learn.microsoft.com/en-us/azure/storage/blobs/immutable-storage-overview) with 7 days retention period will be applied on this container. | ||
|
|
||
| ## Create a Retina Capture | ||
|
|
||
| Retina-operator will create a container blob SAS URL with expiry time determined from Capture Duration to make sure blob SAS URL is invalid soon after the Capture is completed. | ||
| And Retina-operator will update Capture CR with the blob SAL URL to allow the customer read the URL. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.