[AVM Module Issue]: Unable to create private endpoints for Private Link Scope when VNet is in different subscription

[JamesDawson]

Check for previous/existing GitHub issues

• I have checked for previous/existing GitHub issues

Issue Type?

Bug

Module Name

avm/res/insights/private-link-scope

(Optional) Module Version

0.5.2

Description

A deployment failure happens when the Private Link Scope resource is in a different subscription to the Virtual Network.

Generally the error is ResourceGroupNotFound on the private endpoint resource group (since it exists in a different subscription). If a resource group happens to exist with the same name in the PLS subscription, then the deployment produces an InvalidResourceReference.

Private Endpoints must be provisioned in the same subscription as the Virtual Network they are connected to. Whilst this module allows you to customise the resource group for the private endpoint, it currently assumes the same subscription as the Private Link Scope resource.

To avoid needing another parameter, perhaps the scope on this line should infer the subscription from the subnet resource ID?

bicep-registry-modules/avm/res/insights/private-link-scope/main.bicep

Line 160 in 8ee8adb

scope: resourceGroup(privateEndpoint.?resourceGroupName ?? '')

(Optional) Correlation Id

No response

[avm-team-linter]

@JamesDawson, thanks for submitting this issue for the avm/res/insights/private-link-scope module!

Important

A member of the @Azure/avm-res-insights-privatelinkscope-module-owners-bicep or @Azure/avm-res-insights-privatelinkscope-module-contributors-bicep team will review it soon!

[JamesDawson]

I was looking at using the avm/res/key-vault/vault module today and noticed that this module has the same issue:

bicep-registry-modules/avm/res/key-vault/vault/main.bicep

Line 311 in 0cfda08

scope: resourceGroup(privateEndpoint.?resourceGroupName ?? '')

Based on a skim read of the following search results, this seems to be an issue across the board:
https://github.com/search?q=repo%3AAzure%2Fbicep-registry-modules+br%2Fpublic%3Aavm%2Fres%2Fnetwork%2Fprivate-endpoint+language%3ABicep&type=code&l=Bicep

[AlexanderSehr]

Hey folks,
thanks for raising this. I just linked the CoreTeam to it as this is a fundamental issue of the PE Spec and not one module in particular (as was correctly raised).
Without looking to deep into it, if we consider that the PE must be created in the same subscription as the subnet, then the interface should probaly not ask for the ResourceGroupName at all and by default use the one that is defined in the subnetResourceId property.

The implementation would then change to

module keyVault_privateEndpoints 'br/public:avm/res/network/private-endpoint:0.9.0' = [
  for (privateEndpoint, index) in (privateEndpoints ?? []): {
    name: '${uniqueString(deployment().name, location)}-keyVault-PrivateEndpoint-${index}'
    scope: scope: resourceGroup(
    split((privateEndpoint.?subnetResourceId?? '//'), '/')[2],
    split((privateEndpoint.?subnetResourceId?? '////'), '/')[4]
    )
    params: {
      name: privateEndpoint.?name ?? 'pep-${last(split(keyVault.id, '/'))}-${privateEndpoint.?service ?? 'vault'}-${index}'
      (...)
      subnetResourceId: privateEndpoint.subnetResourceId
      location: privateEndpoint.?location ?? reference(
        split(privateEndpoint.subnetResourceId, '/subnets/')[0],
        '2020-06-01',
        'Full'
      ).location
    }
  }
]

Thoughts?

cc: @eriqua, @ChrisSidebotham, @jtracey93 & @ReneHezser for awareness

[ReneHezser]

I think that makes sense (using the resource group of the subnet). Would there be a use-case where the location is a different one?

[JamesDawson]

Thanks @AlexanderSehr. That seems fine, although we would still need optional control over the resource group used to host the private endpoints.

In a scenario where the network resources are managed by a central team, an application team may not have permissions to provision resources alongside the VNet. Therefore being able to specify an alternative resource group name, within the same subscription, would be necessary.

[AlexanderSehr]

Hey @JamesDawson,
thanks for your input. So I take it you'd prefer a solution like

module keyVault_privateEndpoints 'br/public:avm/res/network/private-endpoint:0.9.0' = [
  for (privateEndpoint, index) in (privateEndpoints ?? []): {
    name: '${uniqueString(deployment().name, location)}-keyVault-PrivateEndpoint-${index}'
    scope: !empty(privateEndpoint.?resourceGroupResourceId)
      ? resourceGroup(
          split((privateEndpoint.?resourceGroupResourceId ?? '//'), '/')[2],
          split((privateEndpoint.?resourceGroupResourceId ?? '////'), '/')[4]
        )
      : resourceGroup(
          split((privateEndpoint.?subnetResourceId ?? '//'), '/')[2],
          split((privateEndpoint.?subnetResourceId ?? '////'), '/')[4]
        )
    params: {
      name: privateEndpoint.?name ?? 'pep-${last(split(keyVault.id, '/'))}-${privateEndpoint.?service ?? 'vault'}-${index}'
      (...)
      subnetResourceId: privateEndpoint.subnetResourceId
      location: privateEndpoint.?location ?? reference(
        split(privateEndpoint.subnetResourceId, '/subnets/')[0],
        '2020-06-01',
        'Full'
      ).location
    }
  }
]

In other words: if a resource group resource Id is specified, it will deploy the private endpoint into this resource group (in whatever subscription that resource group is in). If not specified, it will default to deploying the private endpoint into the resource group & subscription of the used virtual network subnet.

[JamesDawson]

Yep that looks good @AlexanderSehr.

[AlexanderSehr]

Yep that looks good @AlexanderSehr.

Great, thanks @JamesDawson.
We just discussed it in the core team and @ReneHezser is working on the interface update.

[AlexanderSehr]

Hey @JamesDawson,
while we're rolling out the above interface, and ask by @ahmadabdalla came up whether we should not rather use the main resources (e.g., Key Vaults) RG as the default location for the PE, and via the already suggested resourceGroupResourceId allow the user to specify a different RG (in a different subscription - as per your requirement).

The subtile change would be from

    scope: resourceGroup(
      split(privateEndpoint.?resourceGroupResourceId ?? privateEndpoint.?subnetResourceId, '/')[2],
      split(privateEndpoint.?resourceGroupResourceId ?? privateEndpoint.?subnetResourceId, '/')[4]
    )

to

    scope: resourceGroup(
      split(privateEndpoint.?resourceGroupResourceId ?? resourceGroup().id, '/')[2],
      split(privateEndpoint.?resourceGroupResourceId ?? resourceGroup().id, '/')[4]
    )

While we'll be discussing this ASAP internally, and while both implementations will statify the requirements and are merely about the 'most common use case' - I'd be curious to hear your thoughts as well.

I'll quote from @ahmadabdalla on this matter

In most scenarios, PEs are deployed alongside their main resource in their own RG vs. the VNET RG. Customer app teams may have subnet join permissions on a centralised VNET in a Landing zone, but may not have permissions to deploy into it. Also considering billing and resource lifecycle perspective.

Ref: #4449

[ahmadabdalla]

Cc @tyconsulting for visibility as he's originally raised this 💪

[JamesDawson]

Thanks @AlexanderSehr, using the resource's RG seems like a sensible default.