Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] add a workflow to do quality checks. #180

Merged
merged 1 commit into from
Jan 5, 2025

Conversation

sayakpaul
Copy link
Collaborator

Follow-up would be a basic test suite (I hope you're okay with that, but if not, I will understand).

@sayakpaul sayakpaul requested a review from a-r-r-o-w January 4, 2025 06:56
Copy link
Owner

@a-r-r-o-w a-r-r-o-w left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is great!

Follow-up would be a basic test suite

Yes, this is perfectly okay with me. To start, let's use dummy checkpoints from hf-internal-testing if possible because using the large original checkpoints will take up to 30 minutes to test everything.

Akin to slow tests in diffusers, we can test bigger checkpoints too, but only when introducing major changes. Small changes and bug fixes just need to ensure the quick small model tests pass. If you have other ideas on how to go about this, would love to hear. No strong opinions here from me

@sayakpaul
Copy link
Collaborator Author

We're perfectly aligned on this!

Yes, this is perfectly okay with me. To start, let's use dummy checkpoints from hf-internal-testing if possible because using the large original checkpoints will take up to 30 minutes to test everything.

Yes 100 percent. This will be our fast tests (i.e., running with dummy checkpoints and as such they will be smoke tests).

Akin to slow tests in diffusers, we can test bigger checkpoints too, but only when introducing major changes. Small changes and bug fixes just need to ensure the quick small model tests pass. If you have other ideas on how to go about this, would love to hear. No strong opinions here from me

Yes! So, for now, I am thinking of just having a combination of a shell script and Makefile that we (you and I) will run manually on GPUs as needed based the criticality of the changes. This will allow us to fire all the commands / runs we need for a relatively quick validation (like the ones you suggested before).

Expect the PRs soon.

Will merge the PR now that I have an approval.

@sayakpaul sayakpaul merged commit b5a5124 into main Jan 5, 2025
1 check passed
@sayakpaul sayakpaul deleted the add-workflow-quality-checks branch January 5, 2025 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants