Skip to content

Commit

Permalink
add example for using local model wtth docker container
Browse files Browse the repository at this point in the history
  • Loading branch information
wirthual committed Feb 10, 2025
1 parent b17b588 commit 1cfad7e
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,17 @@ The cache path inside the docker container is set by the environment variable `H
```
</details>

#### Using local models with Docker container

In order to deploy a local model with a docker container, you need to mount the model inside the container and specify the path in the container to the launch command.

Example:
```bash
cd /tmp
mkdir models && cd models && git clone https://huggingface.co/BAAI/bge-small-en-v1.5
docker run -it -v /tmp/models:/models -p 8081:8081 michaelf34/infinity:latest v2 --model-id "/models/bge-small-en-v1.5" --port 8081
```

#### Advanced CLI usage

<details>
Expand Down

0 comments on commit 1cfad7e

Please sign in to comment.