Connect-AzContext throws the given key 'WriteWarning-System.EventHandler' was not present in the dictionary

[v-bafa]

Description

I have following command for connect Azure with user assigned managed identity in Azure Functions (we have 18 functions running at same time). But got exceptions in some invocations.

Connect-AzAccount -Identity -AccountId $ENV:AZURE_CLIENT_ID

Script or Debug output

ERROR: The given key 'WriteWarning-System.EventHandler`1[[Microsoft.Azure.Commands.Common.Authentication.Abstractions.StreamEventArgs, Microsoft.Azure.PowerShell.Authentication.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]' was not present in the dictionary.

Exception             : 
    Type       : System.Collections.Generic.KeyNotFoundException
    TargetSite : 
        Name          : ThrowKeyNotFoundException
        DeclaringType : System.Collections.Concurrent.ConcurrentDictionary`2[TKey,TValue]
        MemberType    : Method
        Module        : System.Collections.Concurrent.dll
    Message    : The given key 'WriteWarning-System.EventHandler`1[[Microsoft.Azure.Commands.Common.Authentication.Abstractions.StreamEventArgs, Microsoft.Azure.PowerShell.Authentication.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]' was not present in the dictionary.
    Source     : System.Collections.Concurrent
    HResult    : -2146232969
    StackTrace : 
   at System.Collections.Concurrent.ConcurrentDictionary`2.ThrowKeyNotFoundException(TKey key)
   at System.Collections.Concurrent.ConcurrentDictionary`2.get_Item(TKey key)
   at Microsoft.Azure.Commands.Common.Authentication.AzureSession.<>c__DisplayClass86_0`1.<UnregisterComponent>b__0()
   at Microsoft.Azure.Commands.Common.Authentication.AzureSession.UnregisterComponent[T](String componentName)
   at Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand.BeginProcessing()
   at System.Management.Automation.Cmdlet.DoBeginProcessing()
   at System.Management.Automation.CommandProcessorBase.DoBegin()
CategoryInfo          : NotSpecified: (:) [Connect-AzAccount], KeyNotFoundException
FullyQualifiedErrorId : System.Collections.Generic.KeyNotFoundException,Microsoft.Azure.Commands.Profile.ConnectAzureRmAccountCommand
InvocationInfo        : 
    MyCommand        : Connect-AzAccount
    ScriptLineNumber : 30
    OffsetInLine     : 1
    HistoryId        : 1
    ScriptName       : C:\home\site\wwwroot\xxx\run.ps1
    Line             : Connect-AzAccount -Identity -AccountId $ENV:AZURE_CLIENT_ID
                       
    PositionMessage  : At C:\home\site\wwwroot\xxx\run.ps1:30 char:1
                       + Connect-AzAccount -Identity -AccountId $ENV:AZURE_CLIENT_ID
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    PSScriptRoot     : C:\home\site\wwwroot\xxx
    PSCommandPath    : C:\home\site\wwwroot\xxx\run.ps1
    InvocationName   : Connect-AzAccount
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, C:\home\site\wwwroot\xxx\run.ps1: line 30

Environment data

No response

Module versions

No response

Error output

No response

[isra-fel]

Thanks for reporting the issue. I'll follow up with it.

[v-bafa]

Thank you, @isra-fel

I tried to update the command to follows in function script but got following errors when trying to read KeyVault.

if ((Get-AzContext).Count -eq 0) {
    Connect-AzAccount -Identity -AccountId $ENV:AZURE_CLIENT_ID
}

ERROR: Run Connect-AzAccount to login.

Exception :
Type : System.Management.Automation.PSInvalidOperationException
ErrorRecord :
Exception :
Type : System.Management.Automation.ParentContainsErrorRecordException
Message : Run Connect-AzAccount to login.
HResult : -2146233087
CategoryInfo : InvalidOperation: (:) [], ParentContainsErrorRecordException
FullyQualifiedErrorId : InvalidOperation
TargetSite :
Name : get_DefaultContext
DeclaringType : Microsoft.Azure.Commands.ResourceManager.Common.AzureRMCmdlet
MemberType : Method
Module : Microsoft.Azure.PowerShell.Clients.ResourceManager.dll
Message : Run Connect-AzAccount to login.
Source : Microsoft.Azure.PowerShell.Clients.ResourceManager
HResult : -2146233079
StackTrace :
at Microsoft.Azure.Commands.ResourceManager.Common.AzureRMCmdlet.get_DefaultContext()
at Microsoft.Azure.Commands.KeyVault.Models.KeyVaultCmdletBase.get_DataServiceClient()
at Microsoft.Azure.Commands.KeyVault.GetAzureKeyVaultSecret.ExecuteCmdlet()
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.<>c__31.<ExecuteSynchronouslyOrAsJob>b__3_0(T c) at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet, Action1 executor)
at Microsoft.WindowsAzure.Commands.Utilities.Common.CmdletExtensions.ExecuteSynchronouslyOrAsJob[T](T cmdlet)
at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
CategoryInfo : CloseError: (:) [Get-AzKeyVaultSecret], PSInvalidOperationException
FullyQualifiedErrorId : Microsoft.Azure.Commands.KeyVault.GetAzureKeyVaultSecret
InvocationInfo :
MyCommand : Get-AzKeyVaultSecret
ScriptLineNumber : 40
OffsetInLine : 26
HistoryId : 1
ScriptName : C:\home\site\wwwroot\Modules\xxxx\xxxx.psm1
Line : $azureFunctionKey = (Get-AzKeyVaultsecret -VaultName $ENV:ICM_AZFUNCTIONKEY_KV -Name $Env:ICM_AZFUNCTIONKEY_SECRET).SecretValue | ConvertFrom-SecureString -AsPlainText

PositionMessage  : At C:\home\site\wwwroot\Modules\xxxx\xxxx.psm1:40 char:26
                   + … ctionKey = (Get-AzKeyVaultsecret -VaultName $ENV:ICM_AZFUNCTIONKEY_KV …
                   +               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PSScriptRoot     : C:\home\site\wwwroot\Modules\xxxx
PSCommandPath    : C:\home\site\wwwroot\Modules\xxxx\xxxx.psm1
InvocationName   : Get-AzKeyVaultsecret
CommandOrigin    : Internal

ScriptStackTrace : at Write-IcMIncident, C:\home\site\wwwroot\Modules\xxxx\xxxx.psm1: line 40
at Invoke-RemoteScript, C:\home\site\wwwroot\Modules\xxx\xxx.psm1: line 119
at , C:\home\site\wwwroot\yyyyy\run.ps1: line 53
PipelineIterationInfo :

This is the authentication commands in profile.ps1.

if ($env:MSI_SECRET) {
    Connect-AzAccount -Identity -AccountId $ENV:AZURE_CLIENT_ID
}

I added the command to each function as it randomly throws the following errors and failed to read KeyVault.
However, with adding Connect-AzAccount -Identity -AccountId $ENV:AZURE_CLIENT_ID directly, it throws the errors The given key 'WriteWarning-System.EventHandler1` in the issue...after adding the checking line, it failed to read KeyVault randomly again...

We have 16 timer trigger functions within one function app running every 5/10/15 minutes, and 3 queue trigger functions might run at same time. Follows are related configurations.
FUNCTIONS_WORKER_PROCESS_COUNT = 10
FUNCTIONS_WORKER_RUNTIME = powershell
FUNCTIONS_EXTENSION_VERSION = ~4

[boxiao-o]

I have same issue here.
Looks it randomly showed up.

EXCEPTION: Activity function 'DeployActivity' failed: The given key 'WriteInformation-System.EventHandler1[[Microsoft.Azure.Commands.Common.Authentication.Abstractions.StreamEventArgs, Microsoft.Azure.PowerShell.Authentication.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]' was not present in the dictionary.

Exception :
Type : Microsoft.Azure.Functions.PowerShellWorker.Durable.ActivityFailureException
Message : Activity function 'DeployActivity' failed: The given key 'WriteInformation-System.EventHandler`1[[Microsoft.Azure.Commands.Common.Authentication.Abstractions.StreamEventArgs, Microsoft.Azure.PowerShell.Authentication.Abstractions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]' was not present in the dictionary.
HResult : -2146233088
CategoryInfo : NotSpecified: (:) [Invoke-DurableActivity], ActivityFailureException
FullyQualifiedErrorId : Functions.Durable.ActivityFailure,Microsoft.Azure.Functions.PowerShellWorker.Durable.Commands.InvokeDurableActivityCommand
InvocationInfo :
MyCommand : Invoke-DurableActivity
ScriptLineNumber : 13
OffsetInLine : 16
HistoryId : 1
ScriptName : /home/site/wwwroot/DeployOrchestrator/run.ps1
Line : $output += Invoke-DurableActivity -FunctionName 'DeployActivity' -Input $Context.Input

Statement        : Invoke-DurableActivity -FunctionName 'DeployActivity' -Input $Context.Input
PositionMessage  : At /home/site/wwwroot/DeployOrchestrator/run.ps1:13 char:16
                   + …  $output += Invoke-DurableActivity -FunctionName 'DeployActivity' -In …
                   +               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PSScriptRoot     : /home/site/wwwroot/DeployOrchestrator
PSCommandPath    : /home/site/wwwroot/DeployOrchestrator/run.ps1
InvocationName   : Invoke-DurableActivity
CommandOrigin    : Internal

ScriptStackTrace : at , /home/site/wwwroot/DeployOrchestrator/run.ps1: line 13
PipelineIterationInfo :
0
1`

[isra-fel]

Findings / questions

1. What's the root cause of WriteWarning handler being unavailable?
2. If we cannot control (1), we should not expect it to be available, and should not throw when is not there.

[isra-fel]

RCA in #26624