Seqlen of Kernel Benchmark

[Sekri0]

I noticed that the dimension of seqlen (variable M in engine/test.sh) in kernel Benchmark is very small. Does this mean that the test only considers the decode stage and ignores the prefill stage?
I tried to increase M to 4096 and found that the startup of the test became very slow (GPU utilization remained 0 for the first hour).

[lswzjuer]

Sorry, I didn't reply in time due to the National Day holiday. Our solution is mainly for the decoding stage. As mentioned in the paper, the GEMV problem only exists in the decoding stage, so the M value range we tested is 1~16

[Sekri0]

Thanks for the reply, I have one more question. In the end-to-end experiment, which kernel is used in the prefill phase of the w2a8 model