Gangams/logs multi tenancy with service (#1309)

* dcr and cm updates

* fix whitespace

* arm templates for multi-tenancy logs onboarding

* initial implementation

* implement refresh logic

* fix bug

* ignore vuln

* improvements

* handle routing multiple DCRs for a namespace

* refactor code

* cleanup code

* cleanup code

* add telemetry

* bug fixes

* code cleanup

* refactor code

* refactor code

* refactor code

* refactor code

* clean up

* clean up

* clean up

* fix bug

* add dce as dependency for dcr

* remove cm setting until we get the public preview

* add telemetry

* fix bug

* fix config size bug

* fix bug

* fix minor logging

* fix multi-tenancy bug in windows

* fix build error

* fix bug

* fix bug

* minor change

* minor change

* fix bugs

* fix build error

* additional logs

* additional logs

* additional logs

* more logging

* fix bugs

* fix bug

* better naming

* clean up

* fix pr feedback

* fix pr feedback

* minor

* refactor code

* refactor code

* fix bug

* fix bug

* refactor code

* clean up

* use latest api version for dcr,dce and dcra

* add ampls support

* add high log scale mode check for multi-tenancy

* fix minor comment

* fix windows bug

* azmon multi-tenancy conf files

* fix bug

* initial implementation

* fix bug

* add config customizer support

* use defaults

* add support for ama logs service

* yaml updates for ama-logs-multitenancy service and deployment

* minor changes

* fix build error

* update trivy ignore

* fix crash

* bug fixes

* add logging

* set multi-tenancy settings only when enabled

* clean up

* refactor conf

* refactor conf

* conf and yaml updates

* throttle window 5min

* wip

* tenant namespace level config

* revert conf customizer changes

* revert conf customizer changes

* clean up

* clean up

* add service side settings

* clean up

* refactor code

* fix exception

* fix bug

* fix bugs

* add support for default namespace level settings

* refactor code

* clean up code

* handle geneva backward compat

* fix the bug

* fix bug

* fix bug

* add multi-tenancy support for windows

* grafana dashboards

* dashboard updates

* update grafana dashboards

* bump default limits for ama-logs-multitenancy

* grafana dashboards

* grafana dashboard updates

* fix pr feedback

* fix pr feedback

* fix pr feedback

* fix pr feedback

* refactor code

* fix pr feedback

* refactor code

* fix build error

* update trivy ignore

* fix bug in livenessprobe

* fix bug

* add disable throttle config option

* fix bug

* fix bug

* cleanup configmap

* yaml updates

* improve comments in cm

* improve comments in cm

* improve minor comments

* minor update to trigger build

Documentation/DCR/ContainerLogV2Extension.json

@@ -0,0 +1,40 @@
+{
+    "location": "<la-workspace-region>",
+    "properties": {
+        "dataSources": {
+            "extensions": [
+                {
+                    "streams": [
+                        "Microsoft-ContainerLogV2-HighScale"
+                    ],
+                    "extensionName": "ContainerLogV2Extension",
+                    "extensionSettings": {
+                        "dataCollectionSettings": {
+                            "namespaces": ["app-team-k8s-namespace-1", "app-team-k8s-namespace-2"]
+                        }
+                    },
+                    "name": "ContainerLogV2Extension"
+                }
+            ]
+        },
+        "destinations": {
+            "logAnalytics": [
+                {
+                    "workspaceResourceId": "/subscriptions/<subscriptionId>/resourcegroups/<rgName>/providers/microsoft.operationalinsights/workspaces/<workspaceName>",
+                    "name": "la-workspace"
+                }
+            ]
+        },
+        "dataFlows": [
+            {
+                "streams": [
+                    "Microsoft-ContainerLogV2-HighScale"
+                ],
+                "transformKql": "source | where LogLevel in ('CRITICAL', 'ERROR')",
+                "destinations": [
+                    "la-workspace"
+                ]
+            }
+        ]
+    }
+}