Fix vLLM max tokens

[Datta0]

vLLM has max_tokens as Optional[int] but defaulting to 16 here
That means whenever sampling_params is created, it assumes the value 16 and hence this sampling_params.max_tokens ends up being always equal to 16
Then the lighteval benchmark goes on to warn that the output is not in the Gold Format ...

[NathanHB]

Hi ! The behaviour we want is to use the max_new_tokens defined in the task config to be used by default except when it is set in the model's generation config.
In that case, the fix you would need to is set the sampling_params.max_tokens default value to None

[Datta0]

Is there a way to control sampling params while invoking lighteval from the CLI? I couldn't find it in the documentation unfortunately...