Replies: 1 comment
-
Set the WebUI to verbose mode, and look at the parameters it outputs in the console, see if there's anything there different from what you're using in your API call. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello everyone,
I have loaded a local model and it is working very well in the chat tab but when I am using the API it is not speaking so good.
I tried to pass as many arguments that I found but still there are issues.
I also added the same chat_instruct_command but it does not improve.Also greeting is not working. Sometimes it is ok and sometimes it is not ok at all.
What else can I do? Am I missing something?
In model tab there is eager_attention enabled and bf16 but I do not know how to pass them as parameters.
Thank you
Beta Was this translation helpful? Give feedback.
All reactions