Release v3.2.1 into Main

Author: estohlmann

.pre-commit-config.yaml

@@ -23,7 +23,7 @@ repos:
   hooks:
     - id: detect-secrets
       exclude: (?x)^(
-        .*.ipynb|config.yaml
+        .*.ipynb|config.yaml|.*.md
         )$
 
 - repo: https://github.com/pre-commit/pre-commit-hooks

CHANGELOG.md

@@ -1,3 +1,18 @@
+# v3.2.1
+## Bug Fixes
+- Resolved issue where subnet wasn't being passed into ec2 instance creation
+- Resolved role creation issue when deploying with custom subnets
+- Updated docker image to grant permissions on copied in files
+
+## Coming Soon
+- Version 3.3.0 will include a new RAG ingestion pipeline. This will allow users to configure an S3 bucket and an ingestion trigger. When triggered, these documents will be pre-processed and loaded into the selected vector store.
+
+## Acknowledgements
+* @bedanley
+* @estohlmann
+
+**Full Changelog**: https://github.com/awslabs/LISA/compare/v3.2.0...v3.2.1
+
 # v3.2.0
 ## Key Features
 ### Enhanced Deployment Configuration

VERSION

@@ -1 +1 @@
-3.2.0
+3.2.1

ecs_model_deployer/Dockerfile

@@ -1,4 +1,5 @@
 FROM public.ecr.aws/lambda/nodejs:18
 
 COPY ./dist/ ${LAMBDA_TASK_ROOT}
+RUN chmod 777 -R ${LAMBDA_TASK_ROOT}
 CMD ["index.handler"]

lambda/dockerimagebuilder/__init__.py

@@ -70,6 +70,7 @@ def handler(event: Dict[str, Any], context) -> Dict[str, Any]:  # type: ignore [
     try:
         instances = ec2_resource.create_instances(
             ImageId=ami_id,
+            SubnetId=os.environ["LISA_SUBNET_ID"],
             MinCount=1,
             MaxCount=1,
             InstanceType="m5.large",

lib/docs/.vitepress/config.mts

@@ -21,22 +21,34 @@ const navLinks = [
     text: 'System Administrator Guide',
     items: [
       { text: 'What is LISA?', link: '/admin/overview' },
-      { text: 'Architecture Overview', link: '/admin/architecture' },
+      {
+        text: 'Architecture Overview',
+        items: [
+          { text: 'LISA Components', link: '/admin/architecture#lisa-components' },
+        ],
+        link: '/admin/architecture',
+      },
       { text: 'Getting Started', link: '/admin/getting-started' },
+      { text: 'Configure IdP: Cognito & Keycloak Examples', link: '/admin/idp-config' },
       { text: 'Deployment', link: '/admin/deploy' },
-      { text: 'Model Management API Usage', link: '/admin/model-management' },
-      { text: 'Chat UI Configuration', link: '/admin/ui-configuration' },
-      { text: 'API Request Error Handling', link: '/admin/error' },
+      { text: 'Setting Model Management Admin Group', link: '/admin/model-management-admin' },
+      { text: 'LiteLLM', link: '/admin/litellm' },
+      { text: 'API Overview', link: '/admin/api-overview' },
+      { text: 'API Request Error Handling', link: '/admin/api-error' },
+      { text: 'Security', link: '/admin/security' },
     ],
   },
   {
     text: 'Advanced Configuration',
     items: [
-      { text: 'Configuration Schema', link: '/config/configuration' },
+      { text: 'Programmatic API Tokens', link: '/config/api-tokens' },
       { text: 'Model Compatibility', link: '/config/model-compatibility' },
-      { text: 'Rag Vector Stores', link: '/config/vector-stores' },
-      { text: 'Configure IdP: Cognito & Keycloak Examples', link: '/config/idp' },
-      { text: 'LiteLLM', link: '/config/lite-llm' },
+      { text: 'Model Management API', link: '/config/model-management-api' },
+      { text: 'Model Management UI', link: '/config/model-management-ui' },
+      { text: 'Usage & Features', link: '/config/usage' },
+      { text: 'RAG Vector Stores', link: '/config/vector-stores' },
+      { text: 'Branding', link: '/config/branding' },
+      { text: 'Configuration Schema', link: '/config/configuration' },
     ],
   },
   {

lib/docs/admin/error.md lib/docs/admin/api-error.md

@@ -0,0 +1,81 @@
+# API Usage Overview
+
+LISA provides robust API endpoints for managing models, both for users and administrators. These endpoints allow for
+operations such as listing, creating, updating, and deleting models.
+
+## API Gateway and ALB Endpoints
+
+LISA uses two primary APIs for model management:
+
+1. **[User-facing OpenAI-Compatible API](#litellm-routing-in-all-models)**: Available to all users for inference tasks
+   and accessible through the
+   LISA
+   Serve ALB. This API provides an interface for querying and interacting with models deployed on Amazon ECS, Amazon
+   Bedrock, or through LiteLLM.
+2. **[Admin-level Model Management API](/config/model-management-api)**: Available only to administrators through the
+   API Gateway (APIGW). This API
+   allows for full control of model lifecycle management, including creating, updating, and deleting models.
+
+### LiteLLM Routing in All Models
+
+Every model request is routed through LiteLLM, regardless of whether infrastructure (like ECS) is created for it.
+Whether deployed on ECS, external models via Bedrock, or managed through LiteLLM, all models are added to LiteLLM for
+traffic routing. The distinction is whether infrastructure is created (determined by request payloads), but LiteLLM
+integration is consistent for all models. The model management APIs will handle adding or removing model configurations
+from LiteLLM, and the LISA Serve endpoint will handle the inference requests against models available in LiteLLM.
+
+## User-facing OpenAI-Compatible API
+
+The OpenAI-compatible API is accessible through the LISA Serve ALB and allows users to list models available for
+inference tasks. Although not specifically part of the model management APIs, any model that is added or removed from
+LiteLLM via the model management API Gateway APIs will be reflected immediately upon queries to LiteLLM through the LISA
+Serve ALB.
+
+### Listing Models
+
+The `/v2/serve/models` endpoint on the LISA Serve ALB allows users to list all models available for inference in the
+LISA system.
+
+#### Request Example:
+
+```bash
+curl -s -H 'Authorization: Bearer <your_token>' -X GET https://<alb_endpoint>/v2/serve/models
+```
+
+#### Response Example:
+
+```json
+{
+  "data": [
+    {
+      "id": "bedrock-embed-text-v2",
+      "object": "model",
+      "created": 1677610602,
+      "owned_by": "openai"
+    },
+    {
+      "id": "titan-express-v1",
+      "object": "model",
+      "created": 1677610602,
+      "owned_by": "openai"
+    },
+    {
+      "id": "sagemaker-amazon-mistrallite",
+      "object": "model",
+      "created": 1677610602,
+      "owned_by": "openai"
+    }
+  ],
+  "object": "list"
+}
+```
+
+#### Explanation of Response Fields:
+
+These fields are all defined by the OpenAI API specification, which is
+documented [here](https://platform.openai.com/docs/api-reference/models/list).
+
+- `id`: A unique identifier for the model.
+- `object`: The type of object, which is "model" in this case.
+- `created`: A Unix timestamp representing when the model was created.
+- `owned_by`: The entity responsible for the model, such as "openai."

lib/docs/admin/api-overview.md

@@ -120,7 +120,7 @@ This command verifies if the model's weights are already present in your S3 buck
 
 > **WARNING**
 > As of LISA 3.0, the `ecsModels` parameter in `config-custom.yaml` is solely for staging model weights in your S3 bucket.
-> Previously, before models could be managed through the [API](/admin/model-management) or via the Model Management
+> Previously, before models could be managed through the [API](/config/model-management-api) or via the Model Management
 > section of the [Chatbot](/user/chat), this parameter also
 > dictated which models were deployed.
 
@@ -140,13 +140,14 @@ In the `config-custom.yaml` file, configure the `authConfig` block for authentic
 - `jwtGroupsProperty`: Path to the groups field in the JWT token
 - `additionalScopes` (optional): Extra scopes for group membership information
 
-IDP Configuration examples using AWS Cognito and Keycloak can be found: [IDP Configuration Examples](/config/idp)
+IDP Configuration examples using AWS Cognito and Keycloak can be found: [IDP Configuration Examples](/admin/idp-config)
 
 
 ## Step 7: Configure LiteLLM
 We utilize LiteLLM under the hood to allow LISA to respond to the [OpenAI specification](https://platform.openai.com/docs/api-reference).
 For LiteLLM configuration, a key must be set up so that the system may communicate with a database for tracking all the models that are added or removed
-using the [Model Management API](/admin/model-management). The key must start with `sk-` and then can be any arbitrary
+using the [Model Management API](/config/model-management-api). The key must start with `sk-` and then can be any
+arbitrary
 string. We recommend generating a new UUID and then using that as
 the key. Configuration example is below.
 
@@ -229,5 +230,6 @@ services are in the same region as the LISA installation, LISA can use them alon
 
 **Important:** Endpoints or Models statically defined during LISA deployment cannot be removed or updated using the
 LISA Model Management API, and they will not show in the Chat UI. These will only show as part of the OpenAI `/models` API.
-Although there is support for it, we recommend using the [Model Management API](/admin/model-management) instead of the
+Although there is support for it, we recommend using the [Model Management API](/config/model-management-api) instead of
+the
 following static configuration.

lib/docs/admin/getting-started.md

@@ -0,0 +1 @@
+# TODO

lib/docs/config/idp.md lib/docs/admin/idp-config.md

@@ -0,0 +1 @@
+# TODO

lib/docs/admin/litellm.md

@@ -0,0 +1 @@
+# TODO

lib/docs/admin/model-management-admin.md

@@ -0,0 +1 @@
+# TODO