Ollama can't run on 9th gen intel CPU or older, please use a newer version of Ollama

[Ejo2001]

Hello!

I am trying to run Ollama using 2x Intel Arc B580 running in a server with an i7-9700k, but it seems like it can't run with IPEX as the CPU is "Too old". I reported this issue to the Ollama project, and I was directed to submit it here as well. This is the issue I submitted at Ollama: ollama/ollama#9176

Would it be possible to update Ollama version to 0.5.8 or higher?

[qiuxin2012]

We don't have the similar machine like yours. Could you try llama-cpp first? https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md
Our llama.cpp is as the same engine as our ollama.

[JKlesmith]

Fixed the issue by downloading the latest ollama release and copying over libollama-ggml-base.so into the directory and overwriting the correct libollama-ggml-cpu.so for the requisite cpu platform.

[h9j6k]

Fixed the issue by downloading the latest ollama release and copying over libollama-ggml-base.so into the directory and overwriting the correct libollama-ggml-cpu.so for the requisite cpu platform.

Hello, can you elaborate a bit more on this?

I tried copying libggml-base.so and libggml-cpu-skylakex.so from ollama release (0.5.11) to extracted ollama-portable folder and renaming them to libollama-ggml-base.so and libollama-ggml-cpu.so but there is an error saying 'cannot find symbol fp16_sdp'.

[qiuxin2012]

@JKlesmith do you mean using lib/ollama/libggml-base.so in https://github.com/ollama/ollama/releases/download/v0.5.11/ollama-linux-amd64.tgz to inplace ipex-llm's libollama-ggml-cpu.so?

[JKlesmith]

@h9j6k @qiuxin2012

I moved /lib/ollama/libggml-base.so right into the same directory as IPEX (did not change the name of this one). Then overwrote libollama-ggml-cpu.so with /lib/ollama/libggml-cpu-skylakex.so (I used skylake since I'm on a Xeon 61XX platform). I did not modify libollama-ggml-base.so. I used the tgz file that @qiuxin2012 linked to (the v0.5.11 release).

It's working that it doesn't give a SIGILL: illegal instruction error anymore. But I haven't validated if it's using all of the IPEX optimizations.

[h9j6k]

@h9j6k @qiuxin2012

I moved /lib/ollama/libggml-base.so right into the same directory as IPEX (did not change the name of this one). Then overwrote libollama-ggml-cpu.so with /lib/ollama/libggml-cpu-skylakex.so (I used skylake since I'm on a Xeon 61XX platform). I did not modify libollama-ggml-base.so. I used the tgz file that @qiuxin2012 linked to (the v0.5.11 release).

It's working that it doesn't give a SIGILL: illegal instruction error anymore. But I haven't validated if it's using all of the IPEX optimizations.

@JKlesmith Thank you so much! By the way, any idea why mixing the *.so files from two different releases can actually work?

[Ejo2001]

@qiuxin2012 I tried following the llama-cpp guide, but I run into this error:

Setting up intel-i915-dkms (1.23.10.83.231129.91+i127-1) ...
Removing old intel-i915-dkms-1.23.10.83.231129.91 DKMS files...
Deleting module intel-i915-dkms-1.23.10.83.231129.91 completely from the DKMS tree.
Loading new intel-i915-dkms-1.23.10.83.231129.91 DKMS files...
AUXILIARY_BUS is enabled for 6.11.0-18-generic.
Building for 6.11.0-18-generic
Building initial module for 6.11.0-18-generic
AUXILIARY_BUS is enabled for 6.11.0-18-generic.
ERROR: Cannot create report: [Errno 17] File exists: '/var/crash/intel-i915-dkms.0.crash'
Error! Bad return status for module build on kernel: 6.11.0-18-generic (x86_64)
Consult /var/lib/dkms/intel-i915-dkms/1.23.10.83.231129.91/build/make.log for more information.
dpkg: error processing package intel-i915-dkms (--configure):
 installed intel-i915-dkms package post-installation script subprocess returned error exit status 10
Errors were encountered while processing:
 intel-i915-dkms
E: Sub-process /usr/bin/dpkg returned an error code (1)