Deploying on online platforms

[jmn]

Thanks for this lovely project. I am looking to do embeddings fast using an online service of which there are a few. I have tried https://runpod.io and it worked great if I manually installed infinity and started it from a shell but unfortunately I am having some trouble getting up and running using the docker image.

When running with the docker image I get the message: "No such option -i" "Error: "Try 'infinity_emb --help' for help." repeatedly in the logs.

Looking for help successfully running the docker image and tips for services where I can pay by GPU usage while using infinity to do embeddings.

[michaelfeil]

@jmn

Seems like you are specifying some custom arguments when running on runpod.io (I am not familiar with the service of runpod).

The message you are seeing is because you are doing something like infinity_emb -i, where -i is not a recognized command. The fact that you are seeing this message

Troubleshooting guide

1. try he docker run command from the readme wherever you have docker installed.
2. Try to not provide any arguments, and forward port 7797 in the settings of runpod. This should launch bge-small on port 7997.
3. If everyting works, launch with your prefered options, dont go all in at the beginning.

Best
Michael

[jmn]

Thanks! I tried without setting any arguments and that worked.

I am now looking to get infinity working with Runpod serverless following this guide.

I haven't fully looked into it yet but it seems I will have to create a custom docker image.