Skip to content

Commit

Permalink
Merge branch 'wso2:master' into ai
Browse files Browse the repository at this point in the history
  • Loading branch information
Thenujan-Nagaratnam authored Apr 10, 2024
2 parents d5241e1 + 8f01e6d commit a689dba
Show file tree
Hide file tree
Showing 341 changed files with 1,814 additions and 69,277 deletions.
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,12 @@ en/site
.DS_Store

# Ignore node modules
/en/tools/config-catalog-generator/node_modules/
/en/tools/config-catalog-generator/dist/
/en/tools/config-catalog-generator-cc/node_modules/
/en/tools/config-catalog-generator-cc/dist/
/en/tools/config-catalog-generator-mi/node_modules/
/en/tools/config-catalog-generator-mi/dist/

# prefers Python virtual environment for development
venv/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ There are two approaches to configure a secondary user store. It can be configur

5. Enter values for the properties, using the descriptions in the Descriptions column for guidance.

The properties vary based on the user store manager class that you selected, and there may be additional properties under "Optional" or "Advanced" sections at the bottom of the screen. For information on the properties that are used when defining user stores, see [Properties of User Stores](https://docs.wso2.com/display/ADMIN44x/Working+with+Properties+of+User+Stores).
The properties vary based on the user store manager class that you selected, and there may be additional properties under "Optional" or "Advanced" sections at the bottom of the screen. For information on the properties that are used when defining user stores, see [Properties of User Stores]({{base_path}}/administer/managing-users-and-roles/managing-user-stores/working-with-properties-of-user-stores/).

[![Secondary user store definition]({{base_path}}/assets/img/administer/secondary-user-store-definition.png)]({{base_path}}/assets/img/administer/secondary-user-store-definition.png)

Expand Down
2 changes: 1 addition & 1 deletion en/docs/administer/role-based-access-control.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ below scopes chart to define scopes.
| API Categories | apim:api_category, apim:tenantInfo, apim:admin_settings |
| Key Managers | apim:keymanagers_manage, apim:tenantInfo, apim:admin_settings |
| Tasks | apim:api_workflow_view, apim:api_workflow_approve, apim:tenantInfo, apim:admin_setting |
| Settings | apim:app_owner_change, apim:admin_application_view, apim:scope_manage, apim:admin_settings, apim:tenantInfo |
| Settings | apim:app_owner_change, apim:api_provider_change, apim:admin_application_view, apim:scope_manage, apim:admin_settings, apim:tenantInfo |
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Choreo Based Analytics via Proxy

This documentation outlines the steps to configure APIM Analytics with Choreo through a proxy, utilizing a self-signed certificate for secure communication.

## Prerequisites

- Install OpenSSL.

## Installation and Configuration Steps

### Step 1: Install mitmproxy

First, install mitmproxy by following the instructions on the [official mitmproxy documentation](https://docs.mitmproxy.org/stable/overview-installation/). This tool will act as the intercepting proxy between your API Manager and the internet.

### Step 2: Generate a Self-Signed Certificate

To create a secure channel, generate a self-signed certificate using the following steps:

1. Create a configuration file named `req.conf` with the following content:

```toml
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no

[req_distinguished_name]
C = US
ST = VA
L = SomeCity
O = MyCompany
OU = MyDivision
CN = www.company.com

[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
IP.1 = 127.0.0.1
DNS.1 = localhost
DNS.2 = analytics-event-auth.choreo.dev
DNS.3 = analytics-prod-incoming.servicebus.windows.net
```

2. Generate the certificate and key by executing the following command:

```conf
openssl genrsa -out cert.key 2048
openssl req -new -x509 -key cert.key -out cert.crt -config req.conf -extensions 'v3_req'
cat cert.key cert.crt > cert.pem
```

More details on configuring certificates in mitmproxy can be found [here](https://docs.mitmproxy.org/stable/concepts-certificates/).

!!! note
In a production environment, it is crucial to use a certificate issued by a trusted Certificate Authority (CA) instead of a self-signed certificate. This ensures the integrity and trustworthiness of the secure connections established by your infrastructure.

### Step 3: API Manager Configuration

Import the generated cert.crt and cert.key into the API Manager's client-truststore and keystore. Then, apply the following configurations to your deployment.toml file:

```toml
[apim.analytics]
enable = true
config_endpoint = 'https://analytics-event-auth.choreo.dev/auth/v1'
auth_token = 'YOUR_AUTH_TOKEN'

[apim.analytics.properties]
proxy_config_enable = true
proxy_config_host ='127.0.0.1'
proxy_config_port = '3128'
proxy_config_protocol = 'https'
```

Replace `YOUR_AUTH_TOKEN` with the On-premise key. For guidance on obtaining this key, please refer to the instructions provided in the [WSO2 documentation](https://apim.docs.wso2.com/en/latest/api-analytics/choreo-analytics/getting-started-guide/#step-2-register-your-environment).

### Step 3: API Manager Configuration

Launch mitmproxy with the following command to start intercepting traffic:

```conf
mitmweb --web-port 8086 --listen-port 3128 -m regular --no-http2 --certs cert.pem
```

For further details on mitmproxy and its configurations, consult the [mitmproxy documentation](https://docs.mitmproxy.org/stable/overview-installation/).
111 changes: 111 additions & 0 deletions en/docs/api-analytics/on-prem/elk-installation-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ type = "elk"

#### Step 1.2 - Enabling Logs

!!! note
Note that, the following configurations are added by default from the APIM 4.3.0 version onwards.

Open the `wso2am-4.x.x/repository/conf` directory. To enable logging for a reporter, edit the `log4j2.properties` file following the instructions given below.


Expand Down Expand Up @@ -79,6 +82,114 @@ Open the `wso2am-4.x.x/repository/conf` directory. To enable logging for a repor
!!! note
The `apim_metrics.log` file be rolled each day or when the log size reaches the limit of 1000 MB by default. Furthermore, only 10 revisions will be kept and older revisions will be deleted automatically. You can change these configurations by updating the configurations provided in step 2 given above in this. section.

!!! note
Following are the details that are available on analytics log events:

`apim_event_response` - This event will be triggered for each successful API invocation. When a API associated backend returns an error response still it will be logged through this event.

```
apim_event_response:
{
"apiCreator": "admin",
"apiCreatorTenantDomain": "carbon.super",
"apiId": "43d030dc-427f-4678-98e3-87b7d9882b5f",
"apiMethod": "GET",
"apiName": "SampleAPI",
"apiResourceTemplate": "/*",
"apiType": "HTTP",
"apiVersion": "1.0.0",
"applicationId": "2d6c54b0-7c7d-4b50-83dc-e6ae6f88962e",
"applicationName": "DefaultApplication",
"applicationOwner": "admin",
"backendLatency": 13,
"correlationId": "79ed20c3-55b1-434a-adf6-eea25e2d09c3",
"destination": "http://192.168.114.51:8281/services/sampleAPIBackend",
"eventType": "response",
"gatewayType": "SYNAPSE",
"keyType": "SANDBOX",
"platform": "Mac OS X",
"properties":{},
"apiContext":"/api1/2.0.0",
"userName":"[email protected]"
"proxyResponseCode": 202,
"regionId": "default",
"requestMediationLatency": 54,
"requestTimestamp": "2022-01-20T03:34:36.451Z",
"responseCacheHit": false,
"responseLatency": 73,
"responseMediationLatency": 6,
"targetResponseCode": 202,
"userAgent": "Chrome",
"userIp": "172.16.2.70"
}
```

apim_event_faulty - This event will be triggered for each failed and throttled API invocation

```
apim_event_faulty:
{
"apiCreator": "admin",
"apiCreatorTenantDomain": "carbon.super",
"apiId": "43d030dc-427f-4678-98e3-87b7d9882b5f",
"apiName": "SampleAPI",
"apiType": "HTTP",
"apiVersion": "1.0.0",
"applicationId": "0b5ccc91-30e2-4ee5-9355-d1698075c028",
"applicationName": "SampleApp3",
"applicationOwner": "admin",
"correlationId": "ccf2196f-9db8-429b-aaae-98f4c6edf6d7",
"errorCode": 900803,
"errorMessage": "APPLICATION_LEVEL_LIMIT_EXCEEDED",
"errorType": "THROTTLED",
"eventType": "fault",
"gatewayType": "SYNAPSE",
"keyType": "PRODUCTION",
"proxyResponseCode": 429,
"regionId": "default",
"requestTimestamp": "2022-02-01T04:18:48.023Z",
"responseCacheHit": false,
"targetResponseCode": -1
}
```


| **Parameter** | **Type** | **Description** |
|------------------------------|-----------|----------------------------------------------------------------------|
| "apiCreator" | string | The creator of the API. |
| "apiCreatorTenantDomain" | string | The tenant domain of the API creator. |
| "apiId" | string | Unique identifier of the API. |
| "apiMethod" | string | The HTTP method used by the API (e.g., GET, POST). |
| "apiName" | string | The name of the API. |
| "apiResourceTemplate" | string | The template of the API resource accessed. |
| "apiType" | string | The type of the API (e.g., HTTP, REST). |
| "apiVersion" | string | The version of the API. |
| "applicationId" | string | Unique identifier of the application that makes the API call. |
| "applicationName" | string | Name of the application that makes the API call. |
| "applicationOwner" | string | Owner of the application that makes the API call. |
| "backendLatency" | long | The time taken by the backend to process the request. |
| "correlationId" | string | Unique identifier for tracking API calls. |
| "destination" | string | The backend URL to which the API call was redirected. |
| "eventType" | string | The type of event. |
| "gatewayType" | string | The type of the API gateway. |
| "keyType" | string | Indicates whether the API key used was for SANDBOX or PRODUCTION. |
| "platform" | string | Operating system was used to access the API. |
| "properties" | object | Properties of the event. |
| "apiContext" | string | The context of the API call. |
| "userName" | string | The username of the individual who made the API call. |
| "proxyResponseCode" | int | The HTTP response code returned by the API gateway. |
| "regionId" | string | The region identifier for the API call. |
| "requestMediationLatency" | int | Time taken for request mediation. |
| "requestTimestamp" | long | Timestamp when the request was made. |
| "responseCacheHit" | bool | Indicates if the response was served from cache. |
| "responseLatency" | long | Total time taken to respond to the request. |
| "responseMediationLatency" | long | Time taken for response mediation. |
| "targetResponseCode" | int | The HTTP response code received from the backend target. |
| "userAgent" | string | The user agent of the client making the API call. |
| "userIp" | string | The IP address of the user making the API call. |
| "errorCode" | int | The error code generated in a fault. |
| "errorMessage" | string | The error message associated with the fault. |
| "errorType" | string | The type of error (e.g., THROTTLED). |

### Step 2 - Configuring ELK

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/deploy/deploy-api-with-vhost.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/deploy/virtual-host-in-devportal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/design/api-policies/attach-policies-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/design/api-policies/attach-policies-3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/design/api-policies/attach-policies-4.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/design/api-policies/attach-policies-5.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/get_started/api-overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/get_started/qsg-petstore-response.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/get_started/subscriptions-menu.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/get_started/test-api.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/includes/deploy/select-api.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/learn/api-token-copy-generated.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/learn/api-token-create-button.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/learn/api-token-dialog-box.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified en/docs/assets/img/learn/api-token-settings-menu.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed en/docs/assets/img/learn/appId-facebook.png
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added en/docs/assets/img/learn/application-delete.png
Binary file added en/docs/assets/img/learn/application-listing.png
Binary file modified en/docs/assets/img/learn/audit-api-button.png
Binary file modified en/docs/assets/img/learn/changing-api-provider-search.png
Binary file modified en/docs/assets/img/learn/collection-copy-id.png
Binary file modified en/docs/assets/img/learn/collection-create-new.png
Binary file modified en/docs/assets/img/learn/create-facebook-app.png
Binary file modified en/docs/assets/img/learn/graphql-response-query.png
Binary file modified en/docs/assets/img/learn/mediation-json-validator.png
Binary file modified en/docs/assets/img/learn/mediation-regex-policy.png
Binary file modified en/docs/assets/img/learn/mediation-xml-validator.png
Binary file modified en/docs/assets/img/learn/navigate-main-resources.png
Binary file modified en/docs/assets/img/learn/sample-audit-report.png
Binary file modified en/docs/assets/img/learn/select-api.png
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ Follow the instructios below to invoke a **GraphQL Query operation** using the G
1. Enter the following sample query.

```
query{
query testQuery{
human(id:1000){
id
name
Expand Down Expand Up @@ -108,7 +108,7 @@ Follow the instructios below to invoke a **GraphQL Subscription operation** usin
1. Enter the following sample query to execute a subscription operation via WebSockets.
```
subscription {
subscription testSubscription{
reviewAdded(episode: JEDI) {
stars
episode
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ First enable the approve workflow executor for application creation.

1. Sign in to WSO2 API-M Management Console (`https://<Server-Host>:9443/carbon`).

2. Click **Main** --> **Resources** --> **Browse**.
2. Click **Main** --> **Registry** --> **Browse**.

<a href="{{base_path}}/assets/img/learn/wf-extensions-browse.png"><img src="{{base_path}}/assets/img/learn/wf-extensions-browse.png" width="250" height="100"/></a>
<a href="{{base_path}}/assets/img/learn/navigate-main-resources.png"><img src="{{base_path}}/assets/img/learn/navigate-main-resources.png" width="250" height="100"/></a>

3. Go to the `/_system/governance/apimgt/applicationdata/workflow-extensions.xml` resource, click on `Edit as text` to edit the file, disable the Simple Workflow Executor, and enable **Approval Workflow Executor** for application creation.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Adding an Application Deletion Workflow

Attaching a custom workflow to application deletion, enables an admin to approve/reject application deletion requests made for existing applications. Note that only an admin is able to approve/reject an application deletion request.

After application deletion workflow is enabled, when an application deletion request is made, the application workflow status is changed to the `DELETE PENDING` state. In this state, a consumer can still use the application to subscribe to APIs, generate production and sandbox keys until the application deletion is approved. Once the application deletion request is approved the application will be deleted.

### Engaging the Approval Workflow Executor in the API Manager

1. Sign in to the API Manager Management Console (`https://<Server Host>:9443/carbon`) and go to **Browse** under **Registry**.

[![Workflow Extensions Browse]({{base_path}}/assets/img/learn/navigate-main-resources.png)]({{base_path}}/assets/img/learn/navigate-main-resources.png)


2. Open the `/_system/governance/apimgt/applicationdata/workflow-extensions.xml` resource and click **Edit as text**. Disable the `ApplicationDeletionSimpleWorkflowExecutor` and enable `ApplicationDeletionApprovalWorkflowExecutor`.
```
<WorkFlowExtensions>
...
<!--ApplicationDeletion executor="org.wso2.carbon.apimgt.impl.workflow.ApplicationDeletionSimpleWorkflowExecutor"/-->
<ApplicationDeletion executor="org.wso2.carbon.apimgt.impl.workflow.ApplicationDeletionApprovalWorkflowExecutor"/>
...
</WorkFlowExtensions>
```
The application deletion approval workflow executor is now engaged.
3. Sign in to the WSO2 API Developer Portal (`https://<hostname>:<port>/devportal`) and click **Applications**.
[![Applications Listing Tab]({{base_path}}/assets/img/learn/application-listing.png)]({{base_path}}/assets/img/learn/application-listing.png)
4. Click the **Delete** icon under **Actions** column to open the **Delete Application** popup to delete the desired application. Confirm the delete request by clicking the **Delete** button.
[![Application Delete Tab]({{base_path}}/assets/img/learn/application-delete.png)]({{base_path}}/assets/img/learn/application-delete.png)
5. You will see the workflow status as **DELETE PENDING**.
[![Application Delete Before Approval]({{base_path}}/assets/img/learn/application-delete-before-approval.png)]({{base_path}}/assets/img/learn/application-delete-before-approval.png)
6. Sign in to the Admin Portal (`https://<Server Host>:9443/admin`), list all the tasks for Application delete from **Tasks** --> **Application Deletion** and click on approve (or reject) to approve (or reject) the workflow pending request.
[![Application Delete Admin]({{base_path}}/assets/img/learn/application-delete-admin-entry.png)]({{base_path}}/assets/img/learn/application-delete-admin-entry.png)
7. After approving go back to the API Developer Portal Application listing page. The application will be removed.
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ First, enable the application registration workflow.

1. Start WSO2 API Manager and sign in to the APIM management console (`https://<Server Host>:9443/carbon`).

2. Click **Main** --> **Resources** --> **Browse**.
2. Click **Main** --> **Registry** --> **Browse**.

<a href="{{base_path}}/assets/img/learn/wf-extensions-browse.png"><img src="{{base_path}}/assets/img/learn/wf-extensions-browse.png" width="250" height="100"/></a>
<a href="{{base_path}}/assets/img/learn/navigate-main-resources.png"><img src="{{base_path}}/assets/img/learn/navigate-main-resources.png" width="250" height="100"/></a>

3. Go to the `/_system/governance/apimgt/applicationdata/workflow-extensions.xml` resource, disable the Simple Workflow Executor and enable **Approval Workflow Executor** for application registration key generation. You can enable Approve workflow executor for Product keys or Sandbox keys or both by disabling the simple workflow executor and enable approval workflow executor for the ones you need. Please note that this workflow is not applicable for API keys generation.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Follow the instructions below to change the ownership of an application:

`https://<APIM-hostname>:9443/carbon`

2. Create two users named **Chris** and **Kim** with the `Internal/subscriber` role.
2. Create two users named **Chris** and **Kim** with the `Internal/creator` role.
Refer [Create New Users]({{base_path}}/administer/managing-users-and-roles/managing-users#adding-a-new-user) for more information.

3. We can check the current provider by logging to the publisher portal and by looking at the api tiles.
Expand Down
Loading

0 comments on commit a689dba

Please sign in to comment.