2 weeks of work for the big redesign with lots of exicting new features!

Author: vanpelt

.env

@@ -1,3 +1,4 @@
 .DS_Store
 nohup.out
 .cache/
+.env

.gitignore

@@ -18,7 +18,32 @@ OpenUI let's you describe UI using your imagination, then see it rendered live.
 
 ## Running Locally
 
-You can also run OpenUI locally and use models available to [Ollama](https://ollama.com).  [Install Ollama](https://ollama.com/download) and pull a model like [CodeLlama](https://ollama.com/library/codellama), then assuming you have git and python installed:
+OpenUI supports [OpenAI](https://platform.openai.com/api-keys), [Groq](https://console.groq.com/keys), and any model [LiteLLM](https://docs.litellm.ai/docs/) supports such as [Gemini](https://aistudio.google.com/app/apikey) or [Anthropic (Claude)](https://console.anthropic.com/settings/keys).  The following environment variables are optional, but need to be set in your environment for these services to work:
+
+- **OpenAI** `OPENAI_API_KEY`
+- **Groq** `GROQ_API_KEY`
+- **Gemini** `GEMINI_API_KEY`
+- **Anthropic** `ANTHROPIC_API_KEY`
+- **Cohere** `COHERE_API_KEY`
+- **Mistral** `MISTRAL_API_KEY`
+
+You can also use models available to [Ollama](https://ollama.com).  [Install Ollama](https://ollama.com/download) and pull a model like [Llava](https://ollama.com/library/llava).  If Ollama is not running on http://127.0.0.1:11434, you can set the `OLLAMA_HOST` environment variable to the host and port of your Ollama instance.
+
+### Docker (preferred)
+
+The following command would forward the API keys from your current shell environment and tell Docker to use the Ollama instance running on your machine.
+
+```bash
+export ANTHROPIC_API_KEY=xxx
+export OPENAI_API_KEY=xxx
+docker run -n openui -p 7878:7878 -e OPENAI_API_KEY -e ANTHROPIC_API_KEY -e OLLAMA_HOST=http://host.docker.internal:11434 gchr.io/wandb/openui
+```
+
+Now you can goto [http://localhost:7878](http://localhost:7878) and generate new UI's!
+
+### From Source / Python
+
+Assuming you have git and python installed:
 
 > **Note:** There's a .python-version file that specifies **openui** as the virtual env name.  Assuming you have pyenv and pyenv-virtualenv you can run the following from the root of the repository or just run `pyenv local 3.X` where X is the version of python you have installed.
 > ```bash
@@ -38,25 +63,32 @@ export OPENAI_API_KEY=xxx
 python -m openui
 ```
 
-## Groq
+## LiteLLM
 
-To use the super fast [Groq](https://groq.com) models, set `GROQ_API_KEY` to your Groq api key which you can [find here](https://console.groq.com/keys).  To use one of the Groq models, click the settings icon in the sidebar and choose from the list:
+[LiteLLM](https://docs.litellm.ai/docs/) can be used to connect to basically any LLM service available.  We generate a config automatically based on your environment variables.  You can create your own [proxy config](https://litellm.vercel.app/docs/proxy/configs) to override this behavior.  We look for a custom config in the following locations:
 
-<img src="./assets/settings.jpeg" width="500" alt="Select Groq models" />
+1. `litellm-config.yaml` in the current directory
+2. `/app/litellm-config.yaml` when running in a docker container
+3. An arbitrary path specified by the `LITELLM_CONFIG` environment variable
 
-You can also change the default base url used for Groq (if necessary), i.e.
+For example to use a custom config in docker you can run:
 
 ```bash
-export GROQ_BASE_URL=https://api.groq.com/openai/v1
+docker run -n openui -p 7878:7878 -v $(pwd)/litellm-config.yaml:/app/litellm-config.yaml gchr.io/wandb/openui
 ```
 
+## Groq
+
+To use the super fast [Groq](https://groq.com) models, set `GROQ_API_KEY` to your Groq api key which you can [find here](https://console.groq.com/keys).  To use one of the Groq models, click the settings icon in the nav bar.
+
 ### Docker Compose
 
 > **DISCLAIMER:** This is likely going to be very slow.  If you have a GPU you may need to change the tag of the `ollama` container to one that supports it.  If you're running on a Mac, follow the instructions above and run Ollama natively to take advantage of the M1/M2.
 
 From the root directory you can run:
 
 ```bash
+echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 20)" > .env
 docker-compose up -d
 docker exec -it openui-ollama-1 ollama pull llava
 ```
@@ -65,17 +97,6 @@ If you have your OPENAI_API_KEY set in the environment already, just remove `=xx
 
 *If you make changes to the frontend or backend, you'll need to run `docker-compose build` to have them reflected in the service.*
 
-### Docker
-
-You can build and run the docker file manually from the `/backend` directory:
-
-```bash
-docker build . -t wandb/openui --load
-docker run -p 7878:7878 -e OPENAI_API_KEY -e GROQ_API_KEY wandb/openui
-```
-
-Now you can goto [http://localhost:7878](http://localhost:7878)
-
 ## Development
 
 A [dev container](https://github.com/wandb/openui/blob/main/.devcontainer/devcontainer.json) is configured in this repository which is the quickest way to get started.

README.md

@@ -10,9 +10,10 @@
 **/*.egg-info
 **/.DS_Store
 **/build
+**/wandb
+**/*.db
 
 # flyctl launch added from openui/eval/.gitignore
-openui/eval/**/wandb
 openui/eval/**/datasets
 openui/eval/**/components
 fly.toml

backend/.dockerignore

@@ -0,0 +1,74 @@
+#
+name: Create and publish a Docker image
+
+# Configures this workflow to run every time a change is pushed to the branch called `release`.
+on:
+  - workflow_dispatch
+  - push
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+
+jobs:
+  build-and-push-image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+      attestations: write
+      id-token: write
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+      - uses: pnpm/action-setup@v4
+        name: Install pnpm
+        with:
+          version: 9
+          run_install: false
+      - name: Install Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: "pnpm"
+      - name: Get pnpm store directory
+        shell: bash
+        run: |
+          echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
+      - uses: actions/cache@v4
+        name: Setup pnpm cache
+        with:
+          path: ${{ env.STORE_PATH }}
+          key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
+          restore-keys: |
+            ${{ runner.os }}-pnpm-store-
+      - name: Install dependencies
+        run: pnpm install -C frontend
+      # We use npm here because pnpm wasn't executing post hooks
+      - name: Build frontend
+        run: cd frontend && npm run build
+      - name: Log in to the Container registry
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+      - name: Build and push Docker image
+        id: push
+        uses: docker/build-push-action@v6
+        with:
+          context: backend/.
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+      - name: Generate artifact attestation
+        uses: actions/attest-build-provenance@v1
+        with:
+          subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
+          subject-digest: ${{ steps.push.outputs.digest }}
+          push-to-registry: true

backend/.github/workflows/docker.yml

@@ -8,7 +8,7 @@ COPY README.md .
 RUN mkdir -p openui/util && \
     python -m venv /venv && \
     /venv/bin/pip install --upgrade pip setuptools wheel && \
-    /venv/bin/pip install --disable-pip-version-check .
+    /venv/bin/pip install --disable-pip-version-check .[litellm]
 
 # Copy the virtualenv into a distroless image
 FROM python:3.12-slim-bookworm
@@ -22,4 +22,4 @@ WORKDIR /app
 
 RUN pip install --no-deps -U /app
 
-ENTRYPOINT ["python", "-m", "openui"]
+ENTRYPOINT ["python", "-m", "openui", "--litellm"]

backend/Dockerfile

@@ -57,3 +57,10 @@ pytest
 ## Evaluation
 
 The [eval](./openui/eval) folder contains scripts for evaluating the performance of a model.  It automates generating UI, taking screenshots of the UI, then asking `gpt-4-vision-preview` to rate the elements.  More details about the eval pipeline coming soon...
+
+
+## Google Gemeni
+
+```
+gcloud auth application-default login --impersonate-service-account [email protected]
+```

backend/README.md

@@ -2,21 +2,25 @@
 from .logs import setup_logger
 from . import server
 from . import config
+from .litellm import generate_config
 import os
 import uvicorn
 from uvicorn import Config
 import sys
+import subprocess
+import time
+
 
 def is_running_in_docker():
     # Check for the .dockerenv file
-    if os.path.exists('/.dockerenv'):
+    if os.path.exists("/.dockerenv"):
         return True
 
     # Check for Docker-related entries in /proc/self/cgroup
     try:
-        with open('/proc/self/cgroup', 'r') as file:
+        with open("/proc/self/cgroup", "r") as file:
             for line in file:
-                if 'docker' in line:
+                if "docker" in line:
                     return True
     except Exception as e:
         pass
@@ -26,8 +30,17 @@ def is_running_in_docker():
 
     return False
 
+
 if __name__ == "__main__":
     ui = any([arg == "-i" for arg in sys.argv])
+    litellm = (
+        any([arg == "--litellm" for arg in sys.argv])
+        or "OPENUI_LITELLM_CONFIG" in os.environ
+    )
+    # TODO: only render in interactive mode?
+    print(
+        (Path(__file__).parent / "logo.ascii").read_text(), file=sys.stderr, flush=True
+    )
     logger = setup_logger("/tmp/openui.log" if ui else None)
     logger.info("Starting OpenUI AI Server created by W&B...")
 
@@ -63,6 +76,31 @@ def is_running_in_docker():
             logger.info("Running Terminal UI App")
             app.run()
     else:
+        if litellm:
+            config_path = "litellm-config.yaml"
+            if "OPENUI_LITELLM_CONFIG" in os.environ:
+                config_path = os.environ["OPENUI_LITELLM_CONFIG"]
+            elif os.path.exists("/app/litellm-config.yaml"):
+                config_path = "/app/litellm-config.yaml"
+            else:
+                config_path = generate_config()
+
+            logger.info(
+                f"Starting LiteLLM in the background with config: {config_path}"
+            )
+            litellm_process = subprocess.Popen(
+                ["litellm", "--config", config_path],
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+                text=True,
+            )
+            # Ensure litellm stays up for 5 seconds
+            for i in range(5):
+                if litellm_process.poll() is not None:
+                    stdout, stderr = litellm_process.communicate()
+                    logger.error(f"LiteLLM failed to start:\n{stderr}")
+                    break
+                time.sleep(1)
         logger.info("Running API Server")
         mkcert_dir = Path.home() / ".vite-plugin-mkcert"
 

backend/openui/__main__.py

@@ -49,7 +49,17 @@ class Env(Enum):
 AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
 AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")
 BUCKET_NAME = os.getenv("BUCKET_NAME", "openui")
+
+# Cors, if you're hosting the annotator iframe elsewhere, add it here
+CORS_ORIGINS = os.getenv(
+    "OPENUI_CORS_ORIGINS", "https://wandb.github.io,https://localhost:5173"
+).split(",")
+
+# Model providers
+OLLAMA_HOST = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434")
 OPENAI_BASE_URL = os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1")
-OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
+OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "xxx")
 GROQ_BASE_URL = os.getenv("GROQ_BASE_URL", "https://api.groq.com/openai/v1")
-GROQ_API_KEY = os.getenv("GROQ_API_KEY")
+GROQ_API_KEY = os.getenv("GROQ_API_KEY")
+LITELLM_API_KEY = os.getenv("LITELLM_API_KEY", "xxx")
+LITELLM_BASE_URL = os.getenv("LITELLM_BASE_URL", "http://0.0.0.0:4000")

backend/openui/assets/question.svg

@@ -68,6 +68,14 @@ class Component(BaseModel):
     data = JSONField()
 
 
+class Vote(BaseModel):
+    id = BinaryUUIDField(primary_key=True)
+    user = ForeignKeyField(User, backref="votes")
+    component = ForeignKeyField(Component, backref="votes")
+    vote = BooleanField()
+    created_at = DateTimeField()
+
+
 class Usage(BaseModel):
     input_tokens = IntegerField()
     output_tokens = IntegerField()
@@ -105,7 +113,7 @@ def tokens_since(cls, user_id: str, day: datetime.date) -> int:
         )
 
 
-CURRENT_VERSION = "2024-03-12"
+CURRENT_VERSION = "2024-05-14"
 
 
 def alter(schema: SchemaMigration, ops: list[list], version: str) -> bool:
@@ -135,12 +143,19 @@ def perform_migration(schema: SchemaMigration) -> bool:
         )
         if altered:
             perform_migration(schema)
+    if schema.version == "2024-03-12":
+        version = "2024-05-14"
+        database.create_tables([Vote])
+        schema.version = version
+        schema.save()
+        if version != CURRENT_VERSION:
+            perform_migration(schema)
 
 
 def ensure_migrated():
     if not config.DB.exists():
         database.create_tables(
-            [User, Credential, Session, Component, SchemaMigration, Usage]
+            [User, Credential, Session, Component, SchemaMigration, Usage, Vote]
         )
         SchemaMigration.create(version=CURRENT_VERSION)
     else: