Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save BERT model separately #564

Open
zheng-da opened this issue Oct 16, 2023 · 0 comments
Open

Save BERT model separately #564

zheng-da opened this issue Oct 16, 2023 · 0 comments

Comments

@zheng-da
Copy link
Contributor

Currently, we save all model parameters together. This leads to a problem that if the code is changed, we will not be able to load the model. What's even worse is that the failure happens silently.

This problem becomes more serious when we need to fine-tune the BERT model. It is common that we need to fine-tune the BERT model and load the fine-tuned BERT model to train GNN model. It is expensive to fine-tune the BERT model. Once it is tuned, we prefer to keep using it even if the GraphStorm code is changed.

To solve this problem, we should save the BERT model separately from the remaining model parameters.

@classicsong classicsong added this to the 0.2.1 Release Plan. milestone Oct 17, 2023
@classicsong classicsong removed the 0.2.1 label Nov 9, 2023
@classicsong classicsong removed this from the 0.2.1 Release Plan. milestone Nov 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants