Skip to content

Commit

Permalink
refactor: remove transfer-02 sample (#366)
Browse files Browse the repository at this point in the history
* refactor: move transfer-03 to transfer-02

* refactor: rename transfer-06 into transfer-03
  • Loading branch information
ndr-brt authored Feb 13, 2025
1 parent 1da1f44 commit b3e8daa
Show file tree
Hide file tree
Showing 20 changed files with 32 additions and 260 deletions.
2 changes: 1 addition & 1 deletion settings.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ include(":transfer:transfer-05-file-transfer-cloud:cloud-transfer-consumer")
include(":transfer:transfer-05-file-transfer-cloud:cloud-transfer-provider")
include(":transfer:transfer-05-file-transfer-cloud:transfer-file-cloud")

include(":transfer:transfer-06-custom-proxy-data-plane:provider-proxy-data-plane")
include(":transfer:transfer-03-consumer-pull:provider-proxy-data-plane")

include(":transfer:streaming:streaming-01-http-to-http:streaming-01-runtime")
include(":transfer:streaming:streaming-02-kafka-to-http:streaming-02-runtime")
Expand Down
2 changes: 1 addition & 1 deletion system-tests/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ dependencies {
testCompileOnly(project(":transfer:transfer-05-file-transfer-cloud:cloud-transfer-consumer"))
testCompileOnly(project(":transfer:transfer-05-file-transfer-cloud:transfer-file-cloud"))

testCompileOnly(project(":transfer:transfer-06-custom-proxy-data-plane:provider-proxy-data-plane"))
testCompileOnly(project(":transfer:transfer-03-consumer-pull:provider-proxy-data-plane"))

testCompileOnly(project(":federated-catalog:fc-00-basic:fixed-node-resolver"))
testCompileOnly(project(":federated-catalog:fc-01-embedded:fc-connector"))
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@

@EndToEndTest
@Testcontainers
public class Transfer03providerPushTest {
public class Transfer02providerPushTest {

private static final String START_TRANSFER_FILE_PATH = "transfer/transfer-03-provider-push/resources/start-transfer.json";
private static final String START_TRANSFER_FILE_PATH = "transfer/transfer-02-provider-push/resources/start-transfer.json";

@RegisterExtension
static RuntimeExtension provider = getProvider();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@
import static org.hamcrest.Matchers.not;

@EndToEndTest
public class Transfer06customProxyDataPlaneTest {
public class Transfer03httpProxyConsumerPullTest {

private static final String SAMPLE_NAME = "transfer-06-custom-proxy-data-plane";
private static final String SAMPLE_NAME = "transfer-03-consumer-pull";
private static final String START_TRANSFER_FILE_PATH = "transfer/%s/resources/start-transfer.json".formatted(SAMPLE_NAME);

@RegisterExtension
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
@EndToEndTest
public class Transfer04eventConsumerTest {
private static final String CONSUMER_WITH_LISTENER_MODULE_PATH = ":transfer:transfer-04-event-consumer:consumer-with-listener";
private static final String START_TRANSFER_FILE_PATH = "transfer/transfer-02-consumer-pull/resources/start-transfer.json";
private static final String START_TRANSFER_FILE_PATH = "transfer/transfer-03-consumer-pull/resources/start-transfer.json";

@RegisterExtension
static RuntimeExtension provider = getProvider();
Expand Down
17 changes: 6 additions & 11 deletions transfer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,15 @@ Before two connectors can exchange actual data, negotiation has to take place.
The final goal of this example is to showcase the negotiation workflow between two connectors so that
the actual data transfer can take place. This chapter is a prerequisite to the following chapters.

### [Transfer sample 02](./transfer-02-consumer-pull/README.md): Perform a consumer pull exchange between a consumer and a provider
### [Transfer sample 02](transfer-02-provider-push/README.md): Perform a provider push exchange between a consumer and a provider

In this sample you will perform your first actual data transfer.
The purpose of this sample is to show a data exchange between 2 connectors, one representing the
data provider and the other, the consumer. It's based on a "consumer pull" use case that you can find
more details on [Transfer data plane documentation](https://github.com/eclipse-edc/Connector/tree/main/extensions/control-plane/transfer/transfer-data-plane).
This sample demonstrates the "provider push" use case.
More details on [Consumer Pull and Provider Push Transfers](https://eclipse-edc.github.io/documentation/for-adopters/control-plane/#consumer-pull-and-provider-push-transfers).

### [Transfer sample 03](transfer-03-provider-push/README.md): Perform a provider push exchange between a consumer and a provider
### [Transfer sample 03](./transfer-03-consumer-pull): Implement a custom Http Proxy for PULL transfer

This sample demonstrates the "provider push" use case that you can find more details
on [Transfer data plane documentation](https://github.com/eclipse-edc/Connector/tree/main/extensions/control-plane/transfer/transfer-data-plane).
This sample demonstrates how a custom HTTP Proxy Data Plane can be implemented to address PULL transfer type.
More details on [Consumer Pull and Provider Push Transfers](https://eclipse-edc.github.io/documentation/for-adopters/control-plane/#consumer-pull-and-provider-push-transfers).

### [Transfer sample 04](transfer-04-event-consumer/README.md): Consuming connector events

Expand All @@ -46,6 +44,3 @@ a provider that offers a file located in an `Azure Blob Storage`, and a consumer
transfer this file to an `AWS S3 bucket`. Terraform is used for creating all required cloud
resources.

### [Transfer sample 06](./transfer-06-custom-proxy-data-plane): Implement a custom Http Proxy for PULL transfer

This sample demonstrates how a custom HTTP Proxy Data Plane can be implemented to address PULL transfer type.
2 changes: 1 addition & 1 deletion transfer/transfer-01-negotiation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,4 +242,4 @@ The connectors are now ready to enter the data transfer phase.

Note down the `contractAgreementId`. You will need it in the next chapters.

[Next Chapter](../transfer-02-consumer-pull/README.md)
[Next Chapter](../transfer-02-provider-push/README.md)
128 changes: 0 additions & 128 deletions transfer/transfer-02-consumer-pull/README.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ This samples consists of:
The following steps assume your provider and consumer connectors are still up and running and contract
negotiation has taken place successfully. Furthermore, the http server should be up as well.
If not, re-visit the [Prerequisites](../transfer-00-prerequisites/README.md)
, [Negotiation](../transfer-01-negotiation/README.md) and [Consumer Pull](../transfer-02-consumer-pull/README.md) chapters.
, [Negotiation](../transfer-01-negotiation/README.md) and [Consumer Pull](../transfer-03-consumer-pull/README.md) chapters.

# Run the sample

Expand All @@ -43,7 +43,7 @@ You can re-use the same asset, policies and contract negotiation from before.
```bash
curl -X POST "http://localhost:29193/management/v3/transferprocesses" \
-H "Content-Type: application/json" \
-d @transfer/transfer-03-provider-push/resources/start-transfer.json \
-d @transfer/transfer-02-provider-push/resources/start-transfer.json \
-s | jq
```
> keep in mind that, to make a transfer with a provider push method, the dataDestination type should
Expand Down Expand Up @@ -76,4 +76,4 @@ Notice the transfer COMPLETED state
At this step, you can check the data by checking the log of the http server exposed on port 4000, you should see a log
that shows the same data that you can get from https://jsonplaceholder.typicode.com/users.

[Next Chapter](../transfer-04-event-consumer/README.md)
[Next Chapter](../transfer-03-consumer-pull/README.md)
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# Implement a simple "Consumer Pull" Http transfer flow

The purpose of this sample is to show a data exchange between two connectors, one representing the
data provider and the other, the consumer. It's based on a "consumer pull" use case that you can find
more details
data provider and the other, the consumer. You can find more details
on [consumer pull and provider push transfers documentation](https://eclipse-edc.github.io/documentation/for-adopters/control-plane/#consumer-pull-and-provider-push-transfers).

This sample consists of the following steps:
Expand All @@ -12,6 +11,10 @@ This sample consists of the following steps:
* The provider will send an EndpointDataReference to the consumer
* The consumer will call the endpoint and fetch the data

The Http Proxy functionality is not provided by EDC because out there is plenty of FOSS proxy implementations that can
be extended to being able to interact with the EDC, but for example purposes we will implement a really basic Http Proxy
that, please note, it's not meant to be used in production.

## Prerequisites

The following steps assume you already built the base connector in the [Prerequisites](../transfer-00-prerequisites/README.md),
Expand Down Expand Up @@ -64,16 +67,16 @@ order.
### Build the provider connector

```bash
./gradlew transfer:transfer-06-custom-proxy-data-plane:provider-proxy-data-plane:build
./gradlew transfer:transfer-03-consumer-pull:provider-proxy-data-plane:build
```

### Run the connectors

To run the provider, just run the following command

```bash
java -Dedc.fs.config=transfer/transfer-06-custom-proxy-data-plane/resources/configuration/provider.properties \
-jar transfer/transfer-06-custom-proxy-data-plane/provider-proxy-data-plane/build/libs/connector.jar
java -Dedc.fs.config=transfer/transfer-03-consumer-pull/resources/configuration/provider.properties \
-jar transfer/transfer-03-consumer-pull/provider-proxy-data-plane/build/libs/connector.jar
```

To run the consumer, just run the following command (different terminal). Note that the consumer is the same that was built
Expand All @@ -86,7 +89,7 @@ java -Dedc.fs.config=transfer/transfer-00-prerequisites/resources/configuration/

### Negotiate the contract

You can follow the instructions in the [`transfer-01-negotiation`](../transfer-01-negotiation/README.md) to setup a
You can follow the instructions in the [`transfer-01-negotiation`](../transfer-01-negotiation/README.md) to set up a
Contract Negotiation between the two parts.

### Start the transfer
Expand All @@ -98,7 +101,7 @@ Before executing the request, insert the `contractAgreementId` from the previous
```bash
curl -X POST "http://localhost:29193/management/v3/transferprocesses" \
-H "Content-Type: application/json" \
-d @transfer/transfer-06-custom-proxy-data-plane/resources/start-transfer.json \
-d @transfer/transfer-03-consumer-pull/resources/start-transfer.json \
-s | jq
```

Expand Down Expand Up @@ -189,4 +192,5 @@ curl --location --request GET 'http://localhost:19291/public/1' --header 'Author
And the data returned will be the same as in https://jsonplaceholder.typicode.com/users/1

Your first data transfer has been completed successfully.
Continue with the [next chapter](../transfer-03-provider-push/README.md) to run through a "provider push" scenario.

[Next Chapter](../transfer-04-event-consumer/README.md)
Loading

0 comments on commit b3e8daa

Please sign in to comment.