Create roleEligibilityScheduleRequest fails with retries after user creation

[kfwalther]

Describe the bug

Our team is developing an application to create users and assign role eligibilities to those users using the MS Graph SDK. However, we have not been able to get the role eligibility requests to work.

Expected behavior

We expect the roleEligibilityScheduleRequest to complete successfully since the target user object has already been created.

How to reproduce

Here's our C# code to create the user then make the role eligibility request:

...
var client = graphService.GetGraphServiceClient(tenantId);
var user = new User
{
    AccountEnabled = true,
    DisplayName = request.DisplayName,
    MailNickname = request.UserName?.Split('@')[0],
    UserPrincipalName = request.UserName,
    UsageLocation = options.UsageLocation,
    PasswordProfile = new PasswordProfile
    {
        Password = <password>
    }
};
// Add the user account via MS Graph.
var createdUser = await client.Users.PostAsync(user);
// Create the role eligibility request object.
var requestBody = new UnifiedRoleEligibilityScheduleRequest
{
    Action = UnifiedRoleScheduleRequestActions.AdminAssign,
    Justification = $"Assign Global Admin eligibility to user {createdUser.Id}",
    RoleDefinitionId = "62e90394-69f5-4237-9190-012177145e10",   // Global Admin
    DirectoryScopeId = "/",
    PrincipalId = createdUser.Id,
    ScheduleInfo = new RequestSchedule
    {
        StartDateTime = DateTime.UtcNow,
        Expiration = new ExpirationPattern
        {
            Type = ExpirationPatternType.NoExpiration
        },
    },
};
// Send the role eligibility request.
var result = await client.RoleManagement.Directory.RoleEligibilityScheduleRequests.PostAsync(requestBody);

This failed on the last line with an error "The subject is not found". Our suspicion was that the eligibility request was made too quickly, so we added code to use Graph's native retry mechanism to attempt retries:

// Create the role eligibility request object.
var requestBody = new UnifiedRoleEligibilityScheduleRequest
{
    Action = UnifiedRoleScheduleRequestActions.AdminAssign,
    Justification = $"Assign Global Admin eligibility to user {createdUser.Id}",
    RoleDefinitionId = "62e90394-69f5-4237-9190-012177145e10",
    DirectoryScopeId = "/",
    PrincipalId = createdUser.Id,
    ScheduleInfo = new RequestSchedule
    {
        StartDateTime = DateTime.UtcNow,
        Expiration = new ExpirationPattern
        {
            Type = ExpirationPatternType.NoExpiration
        },
    },
};
// Specify retry options to this request. May need to wait some time to make the request to avoid "subject is not found" errors.
var requestOptions = new List<IRequestOption>
{
    new RetryHandlerOption() 
    {
        MaxRetry = 5,
        ShouldRetry = (delay, attempt, httpResponse) =>
        {
            if (httpResponse.StatusCode == System.Net.HttpStatusCode.Unauthorized) 
            {
                return false; 
            }
            if (attempt != numRetries) 
            {
                logger.LogInformation($"Request attempt {attempt + 1} failed, retrying after 3 seconds.");
            }
            else
            {
                logger.LogInformation($"Request attempt {attempt + 1} failed, retrying last attempt in 3 seconds.");
            }
            return true;
        }
    },
};
// Send the role eligibility request.
var result = await client.RoleManagement.Directory.RoleEligibilityScheduleRequests.PostAsync(requestBody, rc => rc.Options = requestOptions);

This version with retries also fails on the last line after about 90 seconds, but with a different error:

Too many retries performed. More than 5 retries encountered while sending the request. (HTTP request failed with status code: NotFound) <binary-junk>

Does this RoleEligibilityScheduleRequests feature work? How else can we troubleshoot this? Do we need a longer "bake time" than 90 seconds for the user object to populate in Graph?

A couple notes:

• The directory roles we're trying to assign (e.g. Global Administrator) have already been "activated" earlier in the code.
• Assigning permanently active directory roles works fine immediately after the user is created:

var directoryObjectReq = new ReferenceCreate
{
    OdataId = graphBaseUrl + "/directoryObjects/" + userObjectId
};
await client.DirectoryRoles[$"roleTemplateId={directoryRoleTemplateId}"].Members.Ref.PostAsync(directoryObjectReq);

SDK Version

5.56.0

Latest version known to work for scenario above?

No response

Known Workarounds

No response

Debug output

Click to expand log

```

</details>


### Configuration

Windows 10 / 11
C# using .NET 8

### Other information

_No response_

[andrueastman]

Thanks for raising this @kfwalther

As this looks to be related to the behaviour of the API and this repo is intended mainly for SDK related issues, any chance you can post the question at the link below to get feedback from the owners of the api?

https://aka.ms/askGraph