Skip to content

Commit 7c50bc5

Browse files
authored
Release v3.2.1 into Main
2 parents 90c3cdf + a4eaba9 commit 7c50bc5

29 files changed

+324
-246
lines changed

.pre-commit-config.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ repos:
2323
hooks:
2424
- id: detect-secrets
2525
exclude: (?x)^(
26-
.*.ipynb|config.yaml
26+
.*.ipynb|config.yaml|.*.md
2727
)$
2828

2929
- repo: https://github.com/pre-commit/pre-commit-hooks

CHANGELOG.md

+15
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,18 @@
1+
# v3.2.1
2+
## Bug Fixes
3+
- Resolved issue where subnet wasn't being passed into ec2 instance creation
4+
- Resolved role creation issue when deploying with custom subnets
5+
- Updated docker image to grant permissions on copied in files
6+
7+
## Coming Soon
8+
- Version 3.3.0 will include a new RAG ingestion pipeline. This will allow users to configure an S3 bucket and an ingestion trigger. When triggered, these documents will be pre-processed and loaded into the selected vector store.
9+
10+
## Acknowledgements
11+
* @bedanley
12+
* @estohlmann
13+
14+
**Full Changelog**: https://github.com/awslabs/LISA/compare/v3.2.0...v3.2.1
15+
116
# v3.2.0
217
## Key Features
318
### Enhanced Deployment Configuration

VERSION

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
3.2.0
1+
3.2.1

ecs_model_deployer/Dockerfile

+1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
FROM public.ecr.aws/lambda/nodejs:18
22

33
COPY ./dist/ ${LAMBDA_TASK_ROOT}
4+
RUN chmod 777 -R ${LAMBDA_TASK_ROOT}
45
CMD ["index.handler"]

lambda/dockerimagebuilder/__init__.py

+1
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ def handler(event: Dict[str, Any], context) -> Dict[str, Any]: # type: ignore [
7070
try:
7171
instances = ec2_resource.create_instances(
7272
ImageId=ami_id,
73+
SubnetId=os.environ["LISA_SUBNET_ID"],
7374
MinCount=1,
7475
MaxCount=1,
7576
InstanceType="m5.large",

lib/docs/.vitepress/config.mts

+20-8
Original file line numberDiff line numberDiff line change
@@ -21,22 +21,34 @@ const navLinks = [
2121
text: 'System Administrator Guide',
2222
items: [
2323
{ text: 'What is LISA?', link: '/admin/overview' },
24-
{ text: 'Architecture Overview', link: '/admin/architecture' },
24+
{
25+
text: 'Architecture Overview',
26+
items: [
27+
{ text: 'LISA Components', link: '/admin/architecture#lisa-components' },
28+
],
29+
link: '/admin/architecture',
30+
},
2531
{ text: 'Getting Started', link: '/admin/getting-started' },
32+
{ text: 'Configure IdP: Cognito & Keycloak Examples', link: '/admin/idp-config' },
2633
{ text: 'Deployment', link: '/admin/deploy' },
27-
{ text: 'Model Management API Usage', link: '/admin/model-management' },
28-
{ text: 'Chat UI Configuration', link: '/admin/ui-configuration' },
29-
{ text: 'API Request Error Handling', link: '/admin/error' },
34+
{ text: 'Setting Model Management Admin Group', link: '/admin/model-management-admin' },
35+
{ text: 'LiteLLM', link: '/admin/litellm' },
36+
{ text: 'API Overview', link: '/admin/api-overview' },
37+
{ text: 'API Request Error Handling', link: '/admin/api-error' },
38+
{ text: 'Security', link: '/admin/security' },
3039
],
3140
},
3241
{
3342
text: 'Advanced Configuration',
3443
items: [
35-
{ text: 'Configuration Schema', link: '/config/configuration' },
44+
{ text: 'Programmatic API Tokens', link: '/config/api-tokens' },
3645
{ text: 'Model Compatibility', link: '/config/model-compatibility' },
37-
{ text: 'Rag Vector Stores', link: '/config/vector-stores' },
38-
{ text: 'Configure IdP: Cognito & Keycloak Examples', link: '/config/idp' },
39-
{ text: 'LiteLLM', link: '/config/lite-llm' },
46+
{ text: 'Model Management API', link: '/config/model-management-api' },
47+
{ text: 'Model Management UI', link: '/config/model-management-ui' },
48+
{ text: 'Usage & Features', link: '/config/usage' },
49+
{ text: 'RAG Vector Stores', link: '/config/vector-stores' },
50+
{ text: 'Branding', link: '/config/branding' },
51+
{ text: 'Configuration Schema', link: '/config/configuration' },
4052
],
4153
},
4254
{
File renamed without changes.

lib/docs/admin/api-overview.md

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# API Usage Overview
2+
3+
LISA provides robust API endpoints for managing models, both for users and administrators. These endpoints allow for
4+
operations such as listing, creating, updating, and deleting models.
5+
6+
## API Gateway and ALB Endpoints
7+
8+
LISA uses two primary APIs for model management:
9+
10+
1. **[User-facing OpenAI-Compatible API](#litellm-routing-in-all-models)**: Available to all users for inference tasks
11+
and accessible through the
12+
LISA
13+
Serve ALB. This API provides an interface for querying and interacting with models deployed on Amazon ECS, Amazon
14+
Bedrock, or through LiteLLM.
15+
2. **[Admin-level Model Management API](/config/model-management-api)**: Available only to administrators through the
16+
API Gateway (APIGW). This API
17+
allows for full control of model lifecycle management, including creating, updating, and deleting models.
18+
19+
### LiteLLM Routing in All Models
20+
21+
Every model request is routed through LiteLLM, regardless of whether infrastructure (like ECS) is created for it.
22+
Whether deployed on ECS, external models via Bedrock, or managed through LiteLLM, all models are added to LiteLLM for
23+
traffic routing. The distinction is whether infrastructure is created (determined by request payloads), but LiteLLM
24+
integration is consistent for all models. The model management APIs will handle adding or removing model configurations
25+
from LiteLLM, and the LISA Serve endpoint will handle the inference requests against models available in LiteLLM.
26+
27+
## User-facing OpenAI-Compatible API
28+
29+
The OpenAI-compatible API is accessible through the LISA Serve ALB and allows users to list models available for
30+
inference tasks. Although not specifically part of the model management APIs, any model that is added or removed from
31+
LiteLLM via the model management API Gateway APIs will be reflected immediately upon queries to LiteLLM through the LISA
32+
Serve ALB.
33+
34+
### Listing Models
35+
36+
The `/v2/serve/models` endpoint on the LISA Serve ALB allows users to list all models available for inference in the
37+
LISA system.
38+
39+
#### Request Example:
40+
41+
```bash
42+
curl -s -H 'Authorization: Bearer <your_token>' -X GET https://<alb_endpoint>/v2/serve/models
43+
```
44+
45+
#### Response Example:
46+
47+
```json
48+
{
49+
"data": [
50+
{
51+
"id": "bedrock-embed-text-v2",
52+
"object": "model",
53+
"created": 1677610602,
54+
"owned_by": "openai"
55+
},
56+
{
57+
"id": "titan-express-v1",
58+
"object": "model",
59+
"created": 1677610602,
60+
"owned_by": "openai"
61+
},
62+
{
63+
"id": "sagemaker-amazon-mistrallite",
64+
"object": "model",
65+
"created": 1677610602,
66+
"owned_by": "openai"
67+
}
68+
],
69+
"object": "list"
70+
}
71+
```
72+
73+
#### Explanation of Response Fields:
74+
75+
These fields are all defined by the OpenAI API specification, which is
76+
documented [here](https://platform.openai.com/docs/api-reference/models/list).
77+
78+
- `id`: A unique identifier for the model.
79+
- `object`: The type of object, which is "model" in this case.
80+
- `created`: A Unix timestamp representing when the model was created.
81+
- `owned_by`: The entity responsible for the model, such as "openai."

lib/docs/admin/getting-started.md

+6-4
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ This command verifies if the model's weights are already present in your S3 buck
120120

121121
> **WARNING**
122122
> As of LISA 3.0, the `ecsModels` parameter in `config-custom.yaml` is solely for staging model weights in your S3 bucket.
123-
> Previously, before models could be managed through the [API](/admin/model-management) or via the Model Management
123+
> Previously, before models could be managed through the [API](/config/model-management-api) or via the Model Management
124124
> section of the [Chatbot](/user/chat), this parameter also
125125
> dictated which models were deployed.
126126
@@ -140,13 +140,14 @@ In the `config-custom.yaml` file, configure the `authConfig` block for authentic
140140
- `jwtGroupsProperty`: Path to the groups field in the JWT token
141141
- `additionalScopes` (optional): Extra scopes for group membership information
142142

143-
IDP Configuration examples using AWS Cognito and Keycloak can be found: [IDP Configuration Examples](/config/idp)
143+
IDP Configuration examples using AWS Cognito and Keycloak can be found: [IDP Configuration Examples](/admin/idp-config)
144144

145145

146146
## Step 7: Configure LiteLLM
147147
We utilize LiteLLM under the hood to allow LISA to respond to the [OpenAI specification](https://platform.openai.com/docs/api-reference).
148148
For LiteLLM configuration, a key must be set up so that the system may communicate with a database for tracking all the models that are added or removed
149-
using the [Model Management API](/admin/model-management). The key must start with `sk-` and then can be any arbitrary
149+
using the [Model Management API](/config/model-management-api). The key must start with `sk-` and then can be any
150+
arbitrary
150151
string. We recommend generating a new UUID and then using that as
151152
the key. Configuration example is below.
152153

@@ -229,5 +230,6 @@ services are in the same region as the LISA installation, LISA can use them alon
229230

230231
**Important:** Endpoints or Models statically defined during LISA deployment cannot be removed or updated using the
231232
LISA Model Management API, and they will not show in the Chat UI. These will only show as part of the OpenAI `/models` API.
232-
Although there is support for it, we recommend using the [Model Management API](/admin/model-management) instead of the
233+
Although there is support for it, we recommend using the [Model Management API](/config/model-management-api) instead of
234+
the
233235
following static configuration.
File renamed without changes.

lib/docs/admin/litellm.md

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# TODO
+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# TODO

lib/docs/admin/security.md

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# TODO
File renamed without changes.

lib/docs/config/branding.md

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# TODO

0 commit comments

Comments
 (0)