ci: update Transformers to v4.49.0 #1282
Draft
+116
−46
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dvrogozh
force-pushed
the
transformers
branch
2 times, most recently
from
62ed9a5 to
fef2644
Compare
|
Transformers v4.48.0 has a regression in tests: I don't see this issue in main at c23a1c193. It might be reasonable to wait for v4.49.0 or bring this issue to HF if it will persist. |
dvrogozh
force-pushed
the
transformers
branch
from
fef2644 to
ce2bbd3
Compare
Same issue observed on v4.49.0. I reported it to Transformers project and will exclude the test manually for now: |
dvrogozh
force-pushed
the
transformers
branch
from
ce2bbd3 to
e0a5875
Compare
dvrogozh
changed the title
dvrogozh
force-pushed
the
transformers
branch
from
e0a5875 to
cf9381b
Compare
dvrogozh
changed the title
dvrogozh
force-pushed
the
transformers
branch
from
cf9381b to
8eea727
Compare
dvrogozh
changed the title
dvrogozh
force-pushed
the
transformers
branch
from
8eea727 to
8294a33
Compare
Changes: * Benchmarking scripts are pruned from Transformers by v4.49.0 due to deprecation. So we don't need to test them anymore. * Some cuda specific tests were generalized to cover non-cuda devices which uncovered some issues. * Some new tests were added which fail for both cuda and xpu. * Few regressions due to changes on Transformers side Fixed tests: * huggingface/transformers@b912f5e * `tests/models/git/test_modeling_git.py::GitModelTest::test_inputs_embeds_matches_input_ids` * huggingface/transformers@b5aaf87 * `tests/pipelines/test_pipelines_video_classification.py::VideoClassificationPipelineTests::test_small_model_pt` * `tests/test_pipeline_mixin.py::VideoClassificationPipelineTests::test_small_model_pt` * huggingface/transformers@42c8ccf * `tests/generation/test_utils.py::GenerationIntegrationTests::test_generated_length_assisted_generation` * huggingface/transformers@9fd123a * `test_model_parallelization` * `test_model_parallel_equal_results` Commits which added new tests (or enabled previously skipped tests) which fail: * huggingface/transformers@23d782e * `tests/pipelines/test_pipelines_text_generation.py::TextGenerationPipelineTests::test_return_dict_in_generate` * `tests/test_pipeline_mixin.py::TextGenerationPipelineTests::test_return_dict_in_generate` * huggingface/transformers@2fa876d * `test_cpu_offload` (some of) * `test_disk_offload_bin` (some of) * `test_disk_offload_safetensors` (some of) * `tests/pipelines/test_pipelines_text_generation.py::TextGenerationPipelineTests::test_small_model_pt_bloom_accelerate` * huggingface/transformers@be2ac09 * `tests/models/paligemma/test_modeling_paligemma.py::PaliGemmaForConditionalGenerationModelTest::test_generate_compilation_all_outputs` * `tests/models/paligemma2/test_modeling_paligemma2.py::PaliGemma2ForConditionalGenerationModelTest::test_generate_compilation_all_outputs` * huggingface/transformers@f19135a * `tests/pipelines/test_pipelines_audio_classification.py::AudioClassificationPipelineTests::test_small_model_pt_fp16` * huggingface/transformers@1fae54c * `tests/trainer/test_trainer.py::TrainerIntegrationPrerunTest::test_gradient_accumulation_loss_alignment_with_model_loss` * huggingface/transformers@15ec971 * `tests/models/qwen2_5_vl/test_processor_qwen2_5_vl.py::Qwen2_5_VLProcessorTest::test_chat_template_video_custom_sampling` * `tests/models/qwen2_5_vl/test_processor_qwen2_5_vl.py::Qwen2_5_VLProcessorTest::test_chat_template_video_special_processing` Regressions: * huggingface/transformers@365fecb * `tests/generation/test_utils.py::GenerationIntegrationTests::test_encoder_decoder_generate_attention_mask` * huggingface/transformers@da334bc * `tests/generation/test_utils.py::GenerationIntegrationTests::test_generate_input_features_as_encoder_kwarg` * huggingface/transformers@bcfc9d7 * `tests/models/llava/test_modeling_llava.py::LlavaForConditionalGenerationModelTest::test_config` * huggingface/transformers#36267 * `tests/utils/test_import_utils.py` * huggingface/transformers#36267 * `tests/models/marian/test_modeling_marian.py` Signed-off-by: Dmitry Rogozhkin <[email protected]>
dvrogozh
force-pushed
the
transformers
branch
from
8294a33 to
3e63839
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes:
Fixed tests:
tests/models/git/test_modeling_git.py::GitModelTest::test_inputs_embeds_matches_input_idstests/pipelines/test_pipelines_video_classification.py::VideoClassificationPipelineTests::test_small_model_pttests/test_pipeline_mixin.py::VideoClassificationPipelineTests::test_small_model_pttests/generation/test_utils.py::GenerationIntegrationTests::test_generated_length_assisted_generationtest_model_parallelizationtest_model_parallel_equal_resultsCommits which added new tests (or enabled previously skipped tests) which fail:
tests/pipelines/test_pipelines_text_generation.py::TextGenerationPipelineTests::test_return_dict_in_generatetests/test_pipeline_mixin.py::TextGenerationPipelineTests::test_return_dict_in_generatetest_cpu_offload(some of)test_disk_offload_bin(some of)test_disk_offload_safetensors(some of)tests/pipelines/test_pipelines_text_generation.py::TextGenerationPipelineTests::test_small_model_pt_bloom_acceleratetests/models/paligemma/test_modeling_paligemma.py::PaliGemmaForConditionalGenerationModelTest::test_generate_compilation_all_outputstests/models/paligemma2/test_modeling_paligemma2.py::PaliGemma2ForConditionalGenerationModelTest::test_generate_compilation_all_outputstests/pipelines/test_pipelines_audio_classification.py::AudioClassificationPipelineTests::test_small_model_pt_fp16tests/trainer/test_trainer.py::TrainerIntegrationPrerunTest::test_gradient_accumulation_loss_alignment_with_model_losstests/models/qwen2_5_vl/test_processor_qwen2_5_vl.py::Qwen2_5_VLProcessorTest::test_chat_template_video_custom_samplingtests/models/qwen2_5_vl/test_processor_qwen2_5_vl.py::Qwen2_5_VLProcessorTest::test_chat_template_video_special_processingRegressions:
tests/generation/test_utils.py::GenerationIntegrationTests::test_encoder_decoder_generate_attention_masktests/generation/test_utils.py::GenerationIntegrationTests::test_generate_input_features_as_encoder_kwargtests/models/llava/test_modeling_llava.py::LlavaForConditionalGenerationModelTest::test_configtests/utils/test_import_utils.pytests/models/marian/test_modeling_marian.pySigned-off-by: Dmitry Rogozhkin [email protected]