Add check for orphaned PR environments

[lorenyu]

I notice in platform-test that there are some orphaned PR environments for closed PRs. I don't know if this is the right solution for this issue, but we could add a check for PR environments that map to orphaned PRs.

Also not sure whether to classify this as a bug or an enhancement

[lorenyu]

@rocketnova @doshitan thoughts on this problem/solution?

[rocketnova]

Hmmm. Here are some initial thoughts:

1. Maybe we could start with defining some guidelines around how long we want to keep environments open on platform-test? I know I've got a few open PRs on platform-test. Some of them are not exactly "orphaned" because I do want to return to the core idea in them, but I am open to us establishing a guideline of not leaving resources running untouched for more than X days (maybe a week?).
2. We could run aws-nuke or cloud-nuke on a regular basis to enforce deletion of resources.
3. If we want to add a check for PR environments that map to orphaned PRs, we would need to establish guidelines on how to map a PR to an environment and define what constitutes an orphaned PR. I'm not opposed to this, but it does seem like more overhead than some of the other ideas we haven't tried yet.

[lorenyu]

• I realized most of the orphaned environments were mine so I cleaned those up.
• I did notice one orphaned workspace t-ptwknr that looks related to the idp work. Can you look into that?

re: 3, I was thinking that an orphaned PR environment is an environment for a PR that's closed. The destroy PR environment script should have destroyed it but it failed for some reason. I noticed a few of those based on my PRs.

[lorenyu]

I think you're talking about something more complex, which is the concept of a stale PR. I think that's a useful discussion too, but perhaps out of scope for this ticket.

[rocketnova]

Ohh great clarification. How would you map closed PRs to terraform environments?

[lorenyu]

The PR environment lives in a separate workspace in the service layer with the name p-<PR number>, so I'm thinking this workflow could do something like:

1. workspaces = terraform -chdir=infra/<APP_NAME>/service workspace list
2. pr_workspaces = filter workspaces to those matching p-(###)
3. orphaned_pr_workspaces = for each PR workspace, use gh pr view <PR_NUMBER> --json state | jq --raw-output .state to see if it is "CLOSED" and get only the closed ones
4. either automatically clean up orphaned_pr_workspaces or notify dev team that these exist