[backfill daemon run retries 3/n] retries of runs in completed backfills should not be considered part of the backfill

[jamiedemaria]

Summary & Motivation

If a run is retried after a backfill is complete, that run is given the backfill tag, but has no affect on the backfill itself. This can cause confusion. Imagine the scenario where a single asset-partition failed in a backfill. The backfill is complete and a user retries the failed asset and the retry succeeds. That retried run will show up in the list of runs for the backfill, but the status for asset will still be failed since the status is locked when the backfill completes. In the UI you can see a successful run for the asset partition, but that the partition is failed

We should be more strict about when run retries are considered part of the backfill. We decided in https://github.com/dagster-io/internal/discussions/12460 that retries that are launched while the backfill is in progress will be part of the backfill, but that retries that are launched after the backfill is complete should not be considered part of the backfill.

To make this change we need to remove the backfill tag from retried runs if the backfill is successful.

How I Tested These Changes

Changelog

Insert changelog entry or delete this section.

[jamiedemaria]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [backfill daemon run retries 3/n] retries of runs in completed backfills should not be considered part of the backfill #25900 👈
• [backfill daemon run retries 2/n] backfill daemon incorporates retries runs when launching new runs #25853
• [backfill daemon run retries 1/n] update how we determine backfill completion to account for retried runs #25771
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jamiedemaria and the rest of your teammates on Graphite