v3.1.0

中文版

新特性

1. 支持swift sample命令进行数据采样，参考这里。
2. 支持强化微调训练，目前已支持拒绝采样微调，参考这里。
3. Grounding任务自定义数据格式重构，参考这里。
4. swift infer支持输出推理速度和ACC/ROUGE/BLEU指标。

新模型

1. Qwen/Qwen2.5-VL-3B-Instruct系列
2. Qwen/Qwen2.5-7B-Instruct-1M系列
3. deepseek-ai/Janus-Pro-1B系列
4. bytedance-research/UI-TARS-2B-SFT系列

新数据集

1. ServiceNow-AI/R1-Distill-SFT
2. bespokelabs/Bespoke-Stratos-17k
3. open-thoughts/OpenThoughts-114k

English Version

New Features

1. Supports the swift sample command for data sampling; refer to here.
2. Supports reinforcement fine-tuning training, with current support for rejection sampling fine-tuning; refer to here.
   3Grounding task custom data format restructuring; refer to here.
3. swift infer supports outputting inference speed and ACC/ROUGE/BLEU metrics.

New Models

1. Qwen/Qwen2.5-VL-3B-Instruct Series
2. Qwen/Qwen2.5-7B-Instruct-1M Series
3. deepseek-ai/Janus-Pro-1B Series
4. bytedance-research/UI-TARS-2B-SFT Series

New Datasets

1. ServiceNow-AI/R1-Distill-SFT
2. bespokelabs/Bespoke-Stratos-17k
3. open-thoughts/OpenThoughts-114k

What's Changed

• add "enable_prefix_caching" args for vllm engine. by @Leoyzen in #2939
• Fix vllm docs link & fix web-ui by @Jintao-Huang in #2970
• Fix sample by @tastelikefeet in #2971
• support merge-lora & quant by @Jintao-Huang in #2973
• support create_checkpoint_symlink by @Jintao-Huang in #2975
• Sampling and RFT by @tastelikefeet in #2977
• support auto dataset mapping by @Jintao-Huang in #2976
• support qwen2_5 long by @Jintao-Huang in #2982
• sys_prompt from file by @lxline in #2980
• support bytedance-research/UI-TARS-2B-SFT series by @Jintao-Huang in #2987
• support Qwen/Qwen2.5-VL-3B-Instruct series model by @Jintao-Huang in #2996
• fix qwen2_5-vl by @Jintao-Huang in #2998
• support Qwen/Qwen2.5-VL-72B-Instruct by @Jintao-Huang in #2999
• refactor grounding by @Jintao-Huang in #3000
• compatible with trl v0.13 by @hjh0119 in #2992
• update R1 dataset by @Jintao-Huang in #3005
• fix qwen2.5-vl grounding (refactor) by @Jintao-Huang in #2979
• fix deploy by @Jintao-Huang in #3007
• support infer metric: acc/rouge or bleu by @Jintao-Huang in #3008
• support deepseek janus pro by @Jintao-Huang in #3009
• update readme by @Jintao-Huang in #3011
• fix parse_dict by @Jintao-Huang in #3012
• update docs by @Jintao-Huang in #3015
• Fix readme & update docs by @Jintao-Huang in #3018
• fix push to hub by @tastelikefeet in #3024
• Fix bugs by @Jintao-Huang in #3025
• fix bugs by @Jintao-Huang in #3026
• Fix qwen tool template to official format by @Leoyzen in #2988
• fix message merging strategy when multi-turn tool calling. by @Leoyzen in #2986

New Contributors

• @Leoyzen made their first contribution in #2939

Full Changelog: v3.0.3...v3.1.0

v3.0.3

中文版

新特性

1. 支持多模态大模型SequenceClassification架构用于多模态分类任务，参考这里。
2. 支持多模态大模型reward model训练。

新模型

1. Shanghai_AI_Laboratory/internlm3-8b-instruct
2. OpenBMB/MiniCPM-o-2_6
3. deepseek-ai/DeepSeek-R1, deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B系列
4. bytedance-research/Valley-Eagle-7B
5. LLM-Research/phi-4
6. Qwen/Qwen2.5-Math-PRM-7B, Qwen/Qwen2.5-Math-PRM-72B
7. MiniMaxAI/MiniMax-Text-01, MiniMaxAI/MiniMax-VL-01

English Version

New Features

1. Support multi-modal large model SequenceClassification architecture for multi-modal classification tasks, see here.
2. Support training of multi-modal reward model.

New Models

1. Shanghai_AI_Laboratory/internlm3-8b-instruct
2. OpenBMB/MiniCPM-o-2_6
3. deepseek-ai/DeepSeek-R1, deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B series
4. bytedance-research/Valley-Eagle-7B
5. LLM-Research/phi-4
6. Qwen/Qwen2.5-Math-PRM-7B, Qwen/Qwen2.5-Math-PRM-72B
7. MiniMaxAI/MiniMax-Text-01, MiniMaxAI/MiniMax-VL-01

What's Changed

• update qlora shell by @Jintao-Huang in #2880
• fix docs by @Jintao-Huang in #2882
• support multi round dpo by @tastelikefeet in #2884
• Support infer n parameter by @tastelikefeet in #2893
• Fix qwen vl eval by @Jintao-Huang in #2892
• fix infer engine by @Jintao-Huang in #2898
• Add phi4 by @tastelikefeet in #2895
• fix link & bug by @Jintao-Huang in #2902
• update video infer examples by @Jintao-Huang in #2840
• Sampler by @tastelikefeet in #2905
• Fix a bug when lint code by @tastelikefeet in #2906
• Fix bugs by @Jintao-Huang in #2907
• update plugin doc by @tastelikefeet in #2908
• fix vllm tp stuck by @Jintao-Huang in #2909
• fix replace_video2image by @Jintao-Huang in #2913
• Fix read file mode by @tastelikefeet in #2915
• fix inspect init by @Jintao-Huang in #2916
• Update rm by @tastelikefeet in #2919
• Add internlm3 dense by @HIT-cwh in #2920
• internlm3 lint pass by @Jintao-Huang in #2923
• Fix web ui log by @tastelikefeet in #2924
• Support Valley by @lxline in #2921
• support minicpm-o by @Jintao-Huang in #2918
• fix vllm tp block by @Jintao-Huang in #2927
• update docs by @Jintao-Huang in #2929
• Support first prms by @tastelikefeet in #2926
• fix Valley by @lxline in #2931
• Support mllm seq_cls/rm by @Jintao-Huang in #2934
• fix bugs by @Jintao-Huang in #2938
• support deepseek-ai/DeepSeek-R1 by @Jintao-Huang in #2940
• Fix quant template by @Jintao-Huang in #2942
• Support minimax by @tastelikefeet in #2943
• Fix mllm seq cls by @Jintao-Huang in #2945
• support deepseek_r1_distill by @Jintao-Huang in #2946
• fix demo_hf by @Jintao-Huang in #2951
• fix infer_stream by @Jintao-Huang in #2952
• fix citest by @Jintao-Huang in #2953
• fix bugs by @Jintao-Huang in #2954
• update requirements by @Jintao-Huang in #2957
• update web-ui images by @tastelikefeet in #2958
• update quant_mllm shell by @Jintao-Huang in #2959
• fix max_length error print by @Jintao-Huang in #2960
• fix seq_cls patcher by @Jintao-Huang in #2963
• ppo compat transformers>=4.47.* by @Jintao-Huang in #2964

Full Changelog: v3.0.2...v3.0.3

v3.0.2

中文版

新特性

1. 支持使用swift app开启可视化推理创空间，参考这里
2. 支持大模型的RM和PPO训练，参考这里
3. 支持SequenceClassification模型（含BERT）的BNB/GPTQ量化，参考这里
4. 支持reward model的推理、部署和BNB/GPTQ量化

新模型

1. ZhipuAI/cogagent-9b-20241220
2. Reward Models: Shanghai_AI_Laboratory/internlm2-1_8b-reward系列, Qwen/Qwen2-Math-RM-72B系列, AI-ModelScope/Skywork-Reward-Llama-3.1-8B系列, AI-ModelScope/GRM_Llama3.1_8B_rewardmodel-ft系列
3. AIDC-AI/Ovis1.6-Gemma2-27B, AIDC-AI/Ovis1.6-Llama3.2-3B
4. PowerInfer/SmallThinker-3B-Preview

新数据集

1. PowerInfer/LONGCOT-Refine-500K, PowerInfer/QWQ-LONGCOT-500K

English Version

New Features

1. Support for using swift app to launch a visual inference creative space, see here
2. Support for RM and PPO training of large models, see here
3. Support for BNB/GPTQ quantization of SequenceClassification models (including BERT), see here
4. Support for inference, deployment, and BNB/GPTQ quantization of reward models

New Models

1. ZhipuAI/cogagent-9b-20241220
2. Reward Models: Shanghai_AI_Laboratory/internlm2-1_8b-reward series, Qwen/Qwen2-Math-RM-72B series, AI-ModelScope/Skywork-Reward-Llama-3.1-8B series, AI-ModelScope/GRM_Llama3.1_8B_rewardmodel-ft series
3. AIDC-AI/Ovis1.6-Gemma2-27B, AIDC-AI/Ovis1.6-Llama3.2-3B
4. PowerInfer/SmallThinker-3B-Preview

New Datasets

1. PowerInfer/LONGCOT-Refine-500K, PowerInfer/QWQ-LONGCOT-500K

What's Changed

• Fix app-ui dropdown by @tastelikefeet in #2787
• fix multi-lora by @Jintao-Huang in #2790
• fix stream infer by @Jintao-Huang in #2793
• fix some web-ui bugs by @tastelikefeet in #2794
• support swift app by @Jintao-Huang in #2792
• fix pt batch infer by @Jintao-Huang in #2800
• fix world_size by @Jintao-Huang in #2801
• update base_model deploy example by @Jintao-Huang in #2803
• fix glm4v by @Jintao-Huang in #2806
• fix swift deploy log error (repeat log) by @Jintao-Huang in #2808
• support ZhipuAI/cogagent-9b-20241220 by @Jintao-Huang in #2810
• fix citest by @Jintao-Huang in #2812
• fix enable_cache by @Jintao-Huang in #2813
• update docs (specific model arguments) by @Jintao-Huang in #2822
• add 'right' option for 'truncation_strategy' by @zsxm1998 in #2754
• Fix glm4v suffix by @Jintao-Huang in #2829
• Update padding side by @Jintao-Huang in #2832
• Update base_to_chat shell by @Jintao-Huang in #2833
• Fix bugs by @Jintao-Huang in #2838
• Fix some bugs by @tastelikefeet in #2848
• support reward_model by @Jintao-Huang in #2849
• Move optimizer to create_optimizer by @tastelikefeet in #2851
• fix post_init by @Jintao-Huang in #2855
• fix cache_name_file by @Jintao-Huang in #2856
• fix telechat template by @Jintao-Huang in #2857
• Update more models by @Jintao-Huang in #2852
• Support quant bert reward by @Jintao-Huang in #2859
• fix jsonl writer by @Jintao-Huang in #2860
• support reward model train by @Jintao-Huang in #2862
• fix vllm video by @Jintao-Huang in #2864
• support mps by @Jintao-Huang in #2866
• Update agent demo by @Jintao-Huang in #2867
• fix bugs by @Jintao-Huang in #2869
• Support ppo by @Jintao-Huang in #2783
• update citest by @Jintao-Huang in #2873
• fix dataset cache bugs by @Jintao-Huang in #2876

New Contributors

• @zsxm1998 made their first contribution in #2754

Full Changelog: v3.0.1...v3.0.2

v3.0.1

中文版

新特性

1. 支持SequenceClassification模型的训练、推理和部署。可以查看以下例子：qwen2.5，bert。
2. LlamaPro支持多模态模型. 例如：qwen2vl、internvl2.5、llama3-vision等。

新模型

1. Qwen/QVQ-72B-Preview
2. iic/DocOwl2
3. OpenGVLab/InternVL2-Pretrain-Models, OpenGVLab/InternVL2_5-4B-AWQ系列, OpenGVLab/InternVL2_5-1B-MPO系列
4. deepseek-ai/DeepSeek-V3系列
5. answerdotai/ModernBERT-base系列
6. AI-ModelScope/paligemma2-3b-pt-224系列, AI-ModelScope/paligemma2-3b-ft-docci-448系列
7. AI-ModelScope/Skywork-o1-Open-Llama-3.1-8B

English Version

New Features:

1. Support for training, inference, and deployment of SequenceClassification models. You can check the following examples: qwen2.5, bert.
2. LlamaPro supports multimodal models, such as qwen2vl, internvl2.5, and llama3-vision.

New Models:

1. Qwen/QVQ-72B-Preview
2. iic/DocOwl2
3. OpenGVLab/InternVL2-Pretrain-Models, OpenGVLab/InternVL2_5-4B-AWQ series, OpenGVLab/InternVL2_5-1B-MPO series
4. deepseek-ai/DeepSeek-V3 series
5. answerdotai/ModernBERT-base series
6. AI-ModelScope/paligemma2-3b-pt-224 series, AI-ModelScope/paligemma2-3b-ft-docci-448 series
7. AI-ModelScope/Skywork-o1-Open-Llama-3.1-8B

What's Changed

• Fix mplug owl2, molmo by @Jintao-Huang in #2724
• fix batch_infer pad_token & florence by @Jintao-Huang in #2725
• Support qwen agent format by @tastelikefeet in #2722
• Support more internvl2.5 awq/mpo & internvl2 pretrain model by @Jintao-Huang in #2726
• support iic/DocOwl2 by @Jintao-Huang in #2728
• update examples by @Jintao-Huang in #2730
• remove files by @Jintao-Huang in #2732
• support paligemma2 by @Jintao-Huang in #2735
• fix windows by @Jintao-Huang in #2733
• support multi-modal llamapro by @tastelikefeet in #2738
• support AI-ModelScope/Skywork-o1-Open-Llama-3.1-8B by @Jintao-Huang in #2739
• Fix windows encoding gbk by @Jintao-Huang in #2741
• fix docs multimodal by @Jintao-Huang in #2742
• support SequenceClassification & update QVQ-72B-Preview by @Jintao-Huang in #2747
• fix web-ui by @Jintao-Huang in #2758
• fix bugs by @Jintao-Huang in #2761
• fix shell by @Jintao-Huang in #2764
• fix app-ui by @tastelikefeet in #2765
• support modern_bert & support bert deploy by @Jintao-Huang in #2767
• fix alpaca by @Jintao-Huang in #2771
• support txt by @Jintao-Huang in #2772
• fix telechat2 template by @Jintao-Huang in #2775
• Fix deepspeed by @Jintao-Huang in #2778
• fix qwen2vl by @Jintao-Huang in #2779
• Fix app ui by @tastelikefeet in #2780
• support deepseek-v3 by @Jintao-Huang in #2781
• Fix app-ui by @tastelikefeet in #2784

Full Changelog: v3.0.0...v3.0.1

v3.0.0

中文版

架构修改与新特性：

具体可以查看这里: https://swift.readthedocs.io/zh-cn/latest/Instruction/ReleaseNote3.0.html

新模型：

1. OpenGVLab/InternVL2_5-1B等系列模型
2. LLM-Research/Llama-3.3-70B-Instruct
3. BAAI/Emu3-Gen
4. deepseek-ai/DeepSeek-V2.5-1210, deepseek-ai/deepseek-vl2等系列模型
5. Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b
6. InfiniAI/Megrez-3b-Instruct, InfiniAI/Megrez-3B-Omni
7. TeleAI/TeleChat2-3B等系列模型

English Version

Architecture Modifications and New Features:

For more details, please visit: https://swift.readthedocs.io/en/latest/Instruction/ReleaseNote3.0.html

New Models:

1. OpenGVLab/InternVL2_5-1B series models
2. LLM-Research/Llama-3.3-70B-Instruct
3. BAAI/Emu3-Gen
4. deepseek-ai/DeepSeek-V2.5-1210, deepseek-ai/deepseek-vl2 series models
5. Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b
6. InfiniAI/Megrez-3b-Instruct, InfiniAI/Megrez-3B-Omni
7. TeleAI/TeleChat2-3B series models

What's Changed

• Refactor All Codes and bump version to 3.0 by @tastelikefeet in #2030
• fix doc by @tastelikefeet in #2545
• fix manifest by @tastelikefeet in #2546
• add doc 2.x by @tastelikefeet in #2548
• fix ui by @tastelikefeet in #2549
• fix infer by @tastelikefeet in #2550
• Refactor mllm by @Jintao-Huang in #2543
• fix ui by @tastelikefeet in #2552
• Fix ui by @tastelikefeet in #2556
• Update ddp infer doc by @Jintao-Huang in #2557
• fix docs by @Jintao-Huang in #2558
• Fix docs by @Jintao-Huang in #2561
• fix log by @tastelikefeet in #2564
• Fix the command line parameter doc by @Jintao-Huang in #2565
• fix context by @Jintao-Huang in #2568
• Documents Updates by @yrk111222 in #2574
• Revert "Documents Updates" by @Jintao-Huang in #2576
• fix hub param by @tastelikefeet in #2572
• Fix bugs by @Jintao-Huang in #2573
• Support internvl2.5 by @Jintao-Huang in #2575
• update english docs by @Jintao-Huang in #2577
• fix en docs by @Jintao-Huang in #2580
• fix docs & add custom example by @Jintao-Huang in #2581
• fix custom example by @Jintao-Huang in #2582
• support llama3.3 by @Jintao-Huang in #2584
• update acc_strategy & fix citest by @Jintao-Huang in #2583
• Support peft0.14 by @tastelikefeet in #2587
• update infer/deploy examples by @Jintao-Huang in #2588
• add image images mapping by @Jintao-Huang in #2594
• update llm sft notebook by @Jintao-Huang in #2599
• fix notebook by @Jintao-Huang in #2600
• Fix streaming by @Jintao-Huang in #2601
• Emu3 gen train by @mi804 in #2602
• compat mllm notebook by @Jintao-Huang in #2604
• Temporarily remove torchacc. by @Jintao-Huang in #2606
• update docs by @Jintao-Huang in #2607
• train and infer scripts for emu3_gen by @mi804 in #2610
• Uodate Document by @yrk111222 in #2615
• update memory usage of emu3-gen by @mi804 in #2611
• move prepare_model by @Jintao-Huang in #2614
• Update mllm notebook by @Jintao-Huang in #2617
• Support all-embedding / all-norm by @Jintao-Huang in #2619
• fix lmdeploy==0.5.* by @Jintao-Huang in #2621
• Support deepseek-ai/DeepSeek-V2.5-1210 by @Jintao-Huang in #2624
• fix use_reentrant gradient_checkpointing by @Jintao-Huang in #2625
• support reward model by @Jintao-Huang in #2628
• fix add_default_tag by @Jintao-Huang in #2631
• fix dataset by @Jintao-Huang in #2636
• fix bugs & update openbuddy models & update docs by @Jintao-Huang in #2638
• fix app-ui by @tastelikefeet in #2641
• Fix post encode by @Jintao-Huang in #2643
• fix bugs by @Jintao-Huang in #2645
• update truncation_strategy by @Jintao-Huang in #2647
• fix swift/Infinity-Instruct by @Jintao-Huang in #2651
• Support LoRA-GA by @lxline in #2650
• support deepseek_vl2 by @Jintao-Huang in #2654
• fix swift/SlimOrca by @Jintao-Huang in #2656
• fix swift/SlimOrca by @Jintao-Huang in #2657
• support Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b:audio by @Jintao-Huang in #2658
• support Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b:base by @Jintao-Huang in #2660
• fix hub by @tastelikefeet in #2661
• fix liger by @tastelikefeet in #2666
• support megrez by @Jintao-Huang in #2667
• fix unsloth resume training by @tastelikefeet in #2668
• fix dataset by @Jintao-Huang in #2670
• Fix bugs by @tastelikefeet in #2671
• fix deepseek_vl2 by @Jintao-Huang in #2675
• support adapters by @Jintao-Huang in #2633
• Support megrez omni by @Jintao-Huang in #2674
• fix docs by @Jintao-Huang in #2679
• fix megrez_omni by @Jintao-Huang in #2680
• fix infer by @Jintao-Huang in #2681
• Fix bugs by @Jintao-Huang in #2687
• Update readme by @Jintao-Huang in #2579
• update wechat by @Jintao-Huang in #2694
• fix readme by @Jintao-Huang in #2696
• Fix web-ui by @tastelikefeet in #2693
• Fix readme by @Jintao-Huang in #2697
• Update banner by @Jintao-Huang in #2699
• fix use_reentrant by @Jintao-Huang in #2700
• update examples by @Jintao-Huang in #2703
• fix eval strategy by @Jintao-Huang in #2707
• Update FAQ by @slin000111 in #2706
• qwen to Qwen by @Jintao-Huang in #2708
• fix timeout & web-ui by @Jintao-Huang in #2709
• Fix multi lora by @tastelikefeet in #2711
• support Qwen/QVQ-72B-Preview by @Jintao-Huang in #2712
• update examples by @Jintao-Huang in #2714
• fix deploy request_config by @Jintao-Huang in #2718
• fix examples by @Jintao-Huang in #2719
• fix gptq group_size by @Jintao-Huang in #2720
• Better error messages by @Jintao-Huang in #2721

New Contributors

• @yrk111222 made their first contribution in #2574
• @lxline made their first contribution in #2650

Full Changelog: v2.6.1...v3.0.0

v2.6.1

New Models:

1. Marco-o1
2. mPLUG-Owl3-7B-241101
3. QwQ-32B-Preview
4. glm-edge, glm-edge-v

New Datasets:

1. OpenO1-SFT

What's Changed

• support part tuner replace_key False by @tastelikefeet in #2438
• bump ms version by @tastelikefeet in #2449
• remove useless code by @tastelikefeet in #2453
• fix qwen2-vl position_ids by @Jintao-Huang in #2461
• fix peft is_multimodal by @Jintao-Huang in #2462
• fix qwen2vl pt infer by @Jintao-Huang in #2463
• [TorchAcc] Update padding strategy when using persistent cache by @eedalong in #2464
• fix kto by @Jintao-Huang in #2478
• Update Common QA by @slin000111 in #2475
• fix awq quant device_map by @Jintao-Huang in #2488
• Fix preprocess num proc by @Jintao-Huang in #2492
• Support marco o1 by @Jintao-Huang in #2496
• fix eval_dataset no by @Jintao-Huang in #2497
• support batch flattening collator by @eedalong in #2499
• fix latex-ocr by @Jintao-Huang in #2510
• support mPLUG-Owl3 241101 by @LukeForeverYoung in #2515
• support qwq by @Jintao-Huang in #2520
• support glm-edge & glm-edge-v by @Jintao-Huang in #2526

New Contributors

• @eedalong made their first contribution in #2464

Full Changelog: v2.6.0...v2.6.1

v2.6.0

English Version

Models

1. Support Qwen2.5 coder models

Feature

1. Correct and support the new loss and gradient accumulation algorithm from transformers.trainer

中文版本

模型

1. 支持千问coder系列模型

功能

1. 支持新的transformers loss和GA计算算法，并修正了其中的bug

What's Changed

• fix gptq ovis quant (inputs_embeds) by @Jintao-Huang in #2378
• [TorchAcc] fix qwen2 for transformers>=4.45 by @baoleai in #2379
• fix trl transformers==4.46 compat by @Jintao-Huang in #2385
• fix deploy stream media_type by @Jintao-Huang in #2393
• fix_mplug_owl3_cut_shape by @Jintao-Huang in #2394
• fix swift deploy (lmdeploy stream) by @Jintao-Huang in #2397
• fix lmdeploy warning & gptq-int4 support by @Jintao-Huang in #2401
• support qwen2.5-coder by @Jintao-Huang in #2400
• fix qwen_vl npu by @Jintao-Huang in #2408
• fix model path by @yingdachen in #2410
• fix qwen_vl dpo by @Jintao-Huang in #2411
• Compat transformers 4.46.2 loss by @Jintao-Huang in #2413
• Fix molmo infer by @Jintao-Huang in #2419
• Fix dataset map by @Jintao-Huang in #2421
• Support qwen2 5 coder series model by @Jintao-Huang in #2422
• fix PPO by @hjh0119 in #2377
• fix docs by @Jintao-Huang in #2425
• compat transformers==4.47 by @Jintao-Huang in #2426
• fix internvl2 lmdeploy>=0.6 deploy by @Jintao-Huang in #2429

Full Changelog: v2.5.2...v2.6.0

v2.5.2

New Models:

1. emu3-chat
2. aya-expanse
3. ministral-8b-inst-2410

New Datasets:

1. llava-video-178k
2. moviechat-1k-test

What's Changed

• fix system by @Jintao-Huang in #2307
• Support Llava-video-178k dataset, a video dataset by @DaozeZhang in #2309
• update molmo readme by @mi804 in #2313
• fix molmo device_map by @Jintao-Huang in #2311
• support emu3-chat by @mi804 in #2322
• Support Moviechat-1k-test dataset, a video datatset by @DaozeZhang in #2319
• fix custom vllm eval args by @Yunnglin in #2325
• fix enable_cache by @Jintao-Huang in #2332
• feat: Add aya models by @Aunali321 in #2335
• compat transformers 4.46 by @Jintao-Huang in #2341
• fix streaming by @tastelikefeet in #2350
• fix doc by @tastelikefeet in #2355
• fix internvl2 eval by @Jintao-Huang in #2362
• fix transformers>=4.46 loss by @Jintao-Huang in #2365
• Support Ministral-8b-inst-2410 by @DaozeZhang in #2371
• fix custom loss by @Jintao-Huang in #2374

New Contributors

• @Aunali321 made their first contribution in #2335

Full Changelog: v2.5.1...v2.5.2

v2.5.1

English Version

New Features:

1. Support for RM for LLM and MLLM, as well as PPO for LLM.

New Models:

1. molmo series
2. mplug-owl3 1b/2b
3. llama3.1-nemotron-70b-instruct
4. deepseek-janus

中文版

新特性：

1. 支持LLM和MLLM的RM, 以及LLM的PPO.

新模型：

1. molmo系列
2. mplug-owl3 1b/2b
3. llama3.1-nemotron-70b-instruct
4. deepseek-janus

What's Changed

• support reward modeling and ppo by @hjh0119 in #2093
• fix rescale_image by @tastelikefeet in #2223
• fix deploy timeout by @Jintao-Huang in #2230
• Fix qwen2 vl batch size by @Jintao-Huang in #2239
• Fix ovis1.6 infer by @Jintao-Huang in #2242
• fix publish by @Jintao-Huang in #2245
• fix qwen2vl video args by @Jintao-Huang in #2251
• Update FAQ by @slin000111 in #2252
• Support molmo series vlm by @mi804 in #2260
• fix sft system by @Jintao-Huang in #2262
• support mplug3 1b/2b by @Jintao-Huang in #2271
• Fix deploy openai by @Jintao-Huang in #2278
• fix vllm ignore suffix by @Jintao-Huang in #2287
• fix lora_target_modules in PPO by @hjh0119 in #2274
• fix quant blocks by @Jintao-Huang in #2292
• Support Llama3.1-nemotron-70b-inst-hf by @DaozeZhang in #2299
• fix ppo citest by @hjh0119 in #2302
• support deepseek-janus by @Jintao-Huang in #2300
• update molmo by @Jintao-Huang in #2305

New Contributors

• @mi804 made their first contribution in #2260

Full Changelog: v2.5.0...v2.5.1

v2.5.0

English Version

New Features:

1. Support for GPTQ & AWQ quantization of multimodal LLMs.
2. Support for dynamic addition of gradient checkpointing in the ViT section to reduce memory consumption.
3. Support for multimodal model pre-training.

New Models:

1. llama3.2, llama3.2-vision series
2. got-ocr2
3. llama3.1-omni
4. ovis1.6-gemma2
5. pixtral-12b
6. telechat2-115b
7. mistral-small-inst-2409

New Datasets:

1. egoschema

中文版

新特性：

1. 支持多模态LLM的gptq&awq量化.
2. 支持动态在vit部分增加gradient_checkpointing, 减少显存消耗.
3. 支持多模态模型预训练.

新模型：

1. llama3.2, llama3.2-vision系列
2. got-ocr2
3. llama3.1-omni
4. ovis1.6-gemma2
5. pixtral-12b
6. telechat2-115b
7. mistral-small-inst-2409

新数据集：

1. egoschema

What's Changed

• fix win32 quote by @tastelikefeet in #2065
• Fix yi template by @Jintao-Huang in #2067
• fix rlhf zero3 by @Jintao-Huang in #2072
• Update qwen2-vl最佳实践.md by @Digital2Slave in #2058
• fix RLHF & max_length by @Jintao-Huang in #2075
• Support Mistral-small-inst-2409 by @DaozeZhang in #2077
• dynamic vit gradient_checkpointing by @Jintao-Huang in #2071
• fix qwen2.5 template by @Jintao-Huang in #2081
• fix multiprocess remove_columns by @Jintao-Huang in #2088
• Support for fine-tuning Pixtral-12B. by @Jintao-Huang in #2090
• fix vllm tokenizer by @Jintao-Huang in #2099
• Fix the issue with media_offset in owl3 when batch_size > 1. by @LukeForeverYoung in #2100
• fix deploy openai compat by @Jintao-Huang in #2101
• fix dataset preprocess by @Jintao-Huang in #2102
• fix cpu infer device_map by @Jintao-Huang in #2103
• fix infer device_map by @Jintao-Huang in #2105
• Support for fine-tuning Llama 3.1 Omni. by @Jintao-Huang in #2106
• support vllm & qwen2-vl video by @Jintao-Huang in #2110
• Fix qwen2-vl zero2/3 by @Jintao-Huang in #2114
• fix qwen2-audio by @Jintao-Huang in #2116
• [TorchAcc] fix: fix find_labels and can_return_loss by @baoleai in #2120
• support got-ocr2 by @Jintao-Huang in #2123
• Support for fine-tuning and deployment of the Llama 3.2 series models. by @Jintao-Huang in #2130
• Support fine-tuning MLLama. by @Jintao-Huang in #2132
• fix not impl bug by @Jintao-Huang in #2134
• Compat vllm & qwen2-vl by @Jintao-Huang in #2136
• fix requirements by @Jintao-Huang in #2137
• fix model_type by @Jintao-Huang in #2138
• fix deploy vllm by @Jintao-Huang in #2141
• fix docs by @Jintao-Huang in #2142
• Fix VLM lora by @tastelikefeet in #2140
• support mllm pt by @Jintao-Huang in #2146
• [TorchAcc] fix: fix save config and additional file for swift and peft by @baoleai in #2149
• update quant_device_map by @Jintao-Huang in #2154
• fix qwen2-audio by @Jintao-Huang in #2157
• fix template by @Jintao-Huang in #2160
• compat trl==0.11 by @Jintao-Huang in #2169
• Support for Egoschema, a new video dataset by @DaozeZhang in #2173
• Update FAQ by @slin000111 in #2165
• fix mplug-owl3 infer by @Jintao-Huang in #2175
• Support quant mllm by @Jintao-Huang in #2177
• update setup.py by @Jintao-Huang in #2205
• fix bugs by @Jintao-Huang in #2207
• support telechat2 by @Jintao-Huang in #2210
• Support ovis 1.6 by @Jintao-Huang in #2211

New Contributors

• @Digital2Slave made their first contribution in #2058
• @LukeForeverYoung made their first contribution in #2100

Full Changelog: v2.4.2...v2.5.0