feat: add stop conversation and regeneration

[NingLu]

Fixes #

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request introduces a new feature that integrates SageMaker models with the chatbot system. It includes the following changes:

• Added a new file source/infrastructure/lib/api/chatbot-management.ts to handle chatbot management functionality.
• Modified source/infrastructure/lib/api/model-management.ts to support SageMaker model integration.
• Added source/lambda/online/common_logic/langchain_integration/models/chat_models/sagemaker_models.py to provide an interface for interacting with SageMaker models.
• Updated various other files across the infrastructure and lambda directories to accommodate the new feature.

The motivation behind this change is to leverage the power of SageMaker's pre-trained models and fine-tuning capabilities to enhance the chatbot's performance and adaptability. This integration will allow for seamless deployment and management of SageMaker models within the chatbot system.

Type of change

• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 29, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/infrastructure/lib/workspace/websocket-api.ts	1 added, 1 removed	The code change removes the import of the Duration module from the aws-cdk-lib library.
source/infrastructure/tsconfig.json	1 added, 1 removed	The code change enables the noUnusedLocals compiler option in TypeScript, which will now flag any unused local variables as errors.
source/infrastructure/package.json	3 added, 1 removed	The code changes involve updating dependencies, adding new AWS SDK dependencies, and removing the aws-sdk dependency.
source/infrastructure/lib/shared/lambda-layers.ts	1 added, 1 removed	The code change removes the unused imports of LayerVersion and Code from the aws-cdk-lib/aws-lambda module.
source/infrastructure/lib/shared/types.ts	2 added, 0 removed	The code change adds two new regions, 'CN_NORTH_1' and 'CN_NORTHWEST_1', to the 'SupportedRegion' enum in the codebase.
source/infrastructure/lib/shared/lambda-helper.ts	2 added, 2 removed	The code changes remove the unnecessary import of the Aws and StackProps classes, and the Architecture class from the AWS Lambda module, while keeping the essential imports.
source/infrastructure/lib/connect/connect-construct.ts	1 added, 2 removed	The code changes involve removing unused imports (AnyPrincipal, Effect, PolicyStatement, LambdaFunction) and optimizing imports for better code organization.
source/infrastructure/lib/api/websocket-api.ts	1 added, 1 removed	The code change removes the import of the Duration class from the aws-cdk-lib library.
source/infrastructure/lib/shared/shared-construct.ts	0 added, 1 removed	The code change removes the import statement for the RemovalPolicy class from the 'aws-cdk-lib' module and imports the 'dotenv' and 'aws-dynamodb' modules.
source/lambda/online/common_logic/langchain_integration/models/chat_models/sagemaker_models.py	67 added, 0 removed	This code defines classes for interacting with SageMaker LLM models, including DeepSeek R1 Distill Llama and Qwen models, with configurations for model parameters, regions, and credentials.
source/infrastructure/bin/main.ts	0 added, 1 removed	The code change removes the declaration of a chatStack variable and only retains the declaration of chatStackOutputs variable.
source/lambda/online/common_logic/common_utils/websocket_utils.py	59 added, 0 removed	The code changes introduce a StopSignalManager class to manage stop signals in a DynamoDB table, allowing the setting, checking, and clearing of stop signals for WebSocket connections, with functions to interact with the manager.
source/lambda/model_management/model_management.py	66 added, 0 removed	The provided code changes introduce new functionality to list SageMaker endpoints based on an optional endpoint type filter, and define endpoint types as constants.
source/infrastructure/lib/ui/ui-stack.ts	3 added, 2 removed	The code changes include removing an unused import (Duration, NestedStack, PhysicalName), adding a deployRegion property to the UserConstruct, and updating the CfnOutput to use userConstruct.userPoolId instead of userConstruct.userPool.userPoolId.
source/infrastructure/lib/chat/chat-stack.ts	6 added, 7 removed	This code change updates the import statements, removes the unused chatbotTableName variable, adds a new stopSignalsTableName property, removes the unused apiLambdaJobSourceLayer, and updates the environment variables passed to the Lambda functions to include the new STOP_SIGNALS_TABLE_NAME.
source/infrastructure/lib/chat/chat-tables.ts	7 added, 0 removed	The code changes introduce a new DynamoDB table named "StopSignals" with a partition key "wsConnectionId" of type string, and update the construct class to include the table name property "stopSignalsTableName".
source/lambda/online/common_logic/common_utils/response_utils.py	23 added, 8 removed	The code changes implement a stop signal mechanism for the WebSocket stream response. It checks for a stop signal before sending each chunk of the response, and if detected, it sends an END message to the frontend, clears the stop signal, and returns the partial answer. It also handles exceptions more robustly by clearing the stop signal and logging errors.
source/infrastructure/cli/magic-config.ts	15 added, 8 removed	The code changes include updating AWS SDK imports to use modular AWS SDK V3 clients and utilities, refactoring the getAwsAccountAndRegion function to use the new AWS SDK V3 clients and utilities, and updating the regular expression pattern for validating the custom domain endpoint.
source/infrastructure/lib/knowledge-base/knowledge-base-stack.ts	10 added, 16 removed	The code changes involve removing the import of Aws and CustomResource from aws-cdk-lib, removing the import of s3deploy from aws-cdk-lib/aws-s3-deployment, and adding a new import of s3Deployment from aws-cdk-lib/aws-s3-deployment. Additionally, it replaces the s3deploy.BucketDeployment with s3Deployment.BucketDeployment and modifies the deployment configuration.
source/infrastructure/lib/user/user-construct.ts	35 added, 22 removed	The code changes involve setting up Cognito resources for user authentication and authorization, including a user pool, domain, client, admin user, and admin group. It also introduces a condition to set up custom OIDC resources for China regions instead of using Cognito. The outputs are modified to include the user pool ID.
source/lambda/online/common_logic/langchain_integration/models/chat_models/init.py	18 added, 7 removed	The code changes include adding support for loading SageMaker models by importing the sagemaker_models module, updating the MODEL_PROVIDER_LOAD_FN_MAP to include SageMaker provider, and improving code readability with better formatting and variable naming.
source/infrastructure/lib/api/chatbot-management.ts	206 added, 0 removed	This code defines an AWS Lambda function and API Gateway resources for managing chatbots, including checking default chatbots, listing indexes, editing chatbots, listing chatbots, and retrieving embeddings.
source/lambda/authorizer/custom_authorizer.py	21 added, 6 removed	The code changes include reorganizing import statements, refactoring the token extraction logic to handle different header formats, constructing the public key using the RSAAlgorithm from the jwt library, and verifying the JWT token signature with additional options like audience, issuer, and verify_exp.
source/infrastructure/lib/ui/ui-exports.ts	13 added, 3 removed	The code changes involve modifying the UiExportsConstruct to create an AWS Custom Resource that puts a JSON configuration file (aws-exports.json) in an S3 bucket during the onCreate and onUpdate lifecycle events, with the file content being the uiProps passed as a prop. It also sets the necessary permissions for the Custom Resource to perform the S3 PutObject action.
source/infrastructure/lib/ui/ui-portal.ts	63 added, 68 removed	This code change replaces the use of the CloudFrontToS3 solution construct with custom CloudFront and S3 configurations, providing more control and customization options for the web portal deployment.
source/infrastructure/lib/model/model-construct.ts	7 added, 89 removed	The code changes:

• Removed imports for unused modules

• Initialized modelRegion from props

• Removed unused sagemakerRole variable

• Removed createKnowledgeBaseEndpointScaling method for endpoint autoscaling

• Simplified initializeSageMakerConfig method |
  | source/infrastructure/lib/api/model-management.ts | 7 added, 6 removed | This code change removes the import and usage of the PythonLayerVersion from the aws-lambda-python-alpha library, and adds a new resource 'endpoints' under the 'modelManagement' resource with a GET method integration to the existing lambda function. |
  | source/lambda/online/common_logic/common_utils/constant.py | 11 added, 7 removed | The code changes include adding/modifying constant values for various classes like IntentType, MKTUserType, HistoryType, LLMTaskType, MessageType, ModelProvider, LLMModelType, SageMakerProvider, Threshold, and introducing a new class WSConnectionSignal with a constant "STOP". |
  | source/infrastructure/lib/api/api-stack.ts | 151 added, 212 removed | This code change modifies the API Gateway setup and adds deployment delay using AWS CDK custom resources. It also refactors the API resource creation into separate constructs for better organization and maintainability. The key changes are:

• Adds a custom resource to introduce a delay during deployment stages.

• Separates API resource creation into separate constructs (ChatbotManagementApi, ChatHistoryApi, PromptApi, IntentionApi, ModelApi).

• Restructures the API Gateway deployment and stage creation.

• Removes unused imports and variables. |