How to test benchmark for Qwen2?

[limertang]

I test benchmark of Qwen2 with qserve_benchmark.py or qserve_e2e_generation.py,it will throw a error:Unsupported model type: QWen2ForCausalLM, how to fix it?

[ys-2020]

Hi! Thank you for your interests in QServe. To benchmark Qwen models, you can modify the model type in the checkpoint config to LlamaForCausalLM as the model architecture of Qwen is similar to Llama. You can also modify this line to support Qwen model family.