[DMS] March release branch, release Thursday 27 #6603

gvasquezvargas · 2025-03-11T19:23:02Z

What Changed?

Created a march release branch for the DMS so we can track all the work together on this branch.

If you want to make edits, create a PR off of this branch.

https://enterprisedb.atlassian.net/browse/DOCS-1270

advocacy_docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/index.mdx

github-actions · 2025-03-12T08:57:04Z

🚀 Deployed on https://deploy-preview-6603--edb-docs-staging.netlify.app

gvasquezvargas

Hey @yidian1997, thanks for the new content! I left a few comments for you, feel free to batch-commit them if you agree.

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

jperozoa

lgtm

mlewandowski-edb

I've finished a partial review of the DMS documentation updates.

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

mlewandowski-edb · 2025-03-19T20:19:49Z

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

+
+- The maximum character length for the ID is 255 characters.
+- You can use lowercase and uppercase characters, numbers, underscores(_) and hyphens(-) for the ID. Other special characters are not supported.
+- The ID must be unique. The target instances cannot have the same ID.


What does this mean?

The target instances cannot have the same ID.

I think I meant to say "Two target instances cannot have the same ID", but thinking about this more, I think we can get rid of the second sentence altogether, WDYT?

I think the second sentence should be removed. However, I wonder if we need to be clearer about in what context the ID should be unique. I think ID must be unique from the perspective that it must not be the same as the name of any other currently running writers. Presumably readers have this same uniqueness constraint across other running readers.

@jperozoa or @yidian1997 can readers and writers have the same ID? Also, is uniqueness required across an entire HCP deployment or is uniqueness only required within the context of an HCP project? In other words, would it be accurate to say that two currently running readers or two currently running writers can have the same ID if they are in different projects? Or would it be accurate to say the no two currently running readers and no two currently running writers attached to the HCP, independent of the HCP project, can have the same ID?

mlewandowski-edb · 2025-03-19T20:29:08Z

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

+The Apicurio client uses it to perform mTLS authentication with the `transporter-rw-service`.
+
+### `AWS_ACCESS_KEY_ID`
+1) If you are using AWS S3, set this value to an access key ID that has read and write permissions for the S3 bucket.


Is this a setting for something that is in a user's environment or in the HCP deployment environment? If the latter, is this a value that a user of the HCP will know or will they have to get this from an administrator? This should be clearer. And, if it is something that is specified in the HCP configuration, the admin's, who may not have set up the HCP, will need to know how find it.

@jperozoa @yidian1997 Settings like this and the RW_SERVICE_HOST have me wondering if there is a way for a user to have an environment specific run-cdcreader.sh and run-cdcwriter.sh files generated for them that would populate some of these values for them based on the HCP deployment.

I think those access key related values should get from administrator.

Who can we ask more about this? If an admin needs to provide this information, where does he/she get it?

Do they check in their AWS console? Is there a command they can run to fetch this and all subsequent values that start with AWS_? Or maybe all these values are related to something established during the installation of the HCP?

Also, in a conversation I had with @jperozoa earlier today, it's my understanding that these S3 related settings are only required if you have large rows in your source database that needs to be migrated. As I understand it, transporter can actually migrate the large rows, but setting these variables will improve performance of migrating large rows. If they are optional, we should note that they are optional but recommended and provide their purpose/benefit.

mlewandowski-edb · 2025-03-19T20:31:52Z

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

+2) If you are using S3-compatible storage, like MinIO, set this value to the access key for your object storage.
+
+### `AWS_SECRET_ACCESS_KEY`
+1) If you are using AWS S3, set this value to a secret access key that has read and write permissions for the specified S3 bucket.


I have the same question here as I did in my comment under to the AWS_ACCESS_KEY_ID setting.

advocacy_docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/index.mdx

mlewandowski-edb · 2025-03-19T20:48:50Z

...l/data-migration-service/getting_started/installing2/linux_x86_64/edb-dms-writer_sles_15.mdx

@@ -29,8 +29,7 @@ Before you begin the installation process:

 - Activate the required SUSE module:
  ```shell
-  sudo SUSEConnect -p PackageHub/12.5/x86_64
-  sudo SUSEConnect -p sle-sdk/12.5/x86_64
+  sudo SUSEConnect -p PackageHub/15.4/x86_64


I wonder if this is correct? Does the .4 correspond to the Service Pack number? If so, I think SLES 15 SP 6 is the version we now support.

@yidian1997 is Matt's suggestions correct?

Thanks for the comments, today I took some time to investigate this and found that the current doc steps are different from the current instructions shown inthe EDB repository. I need ask some questions to the foundation team

The question I asked in the foundation: https://edb.slack.com/archives/C0413N9LGVB/p1742890896071419

...s/edb-postgres-ai/migration-etl/data-migration-service/getting_started/installing2/index.mdx

mlewandowski-edb · 2025-03-19T20:52:44Z

...data-migration-service/getting_started/installing2/linux_x86_64/edb-dms-writer_ubuntu_20.mdx

@@ -0,0 +1,36 @@
+---
+navTitle: Ubuntu 20.04


I believe Ubuntu 20.04 has reached End of Life and we will no longer be supporting it by the end of March 2025. However, the plan is to start to support Ubuntu 24.04 this quarter (Q1 2025).

So, I think this page is not needed, but a 24.04 page will be needed.

Got it, will remove in the next commit.

...s-ai/migration-etl/data-migration-service/getting_started/installing2/linux_x86_64/index.mdx

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

mlewandowski-edb

Finished reviewing remaining files and files that have been changed since my last review.
Sorry for all the comments on references to Ubuntu 20, which is out of support. I suspect there is a plan to take care of them, but I didn't want to assume all the references were well known.

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_reader.mdx

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

mlewandowski-edb · 2025-03-25T17:27:25Z

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

+The Apicurio client uses it to perform mTLS authentication with the `transporter-rw-service`.
+
+### `AWS_ACCESS_KEY_ID`
+1) If you are using AWS S3, set this value to an access key ID that has read and write permissions for the S3 bucket.


Also, in a conversation I had with @jperozoa earlier today, it's my understanding that these S3 related settings are only required if you have large rows in your source database that needs to be migrated. As I understand it, transporter can actually migrate the large rows, but setting these variables will improve performance of migrating large rows. If they are optional, we should note that they are optional but recommended and provide their purpose/benefit.

...cs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/installing/index.mdx

...es-ai/migration-etl/data-migration-service/getting_started/installing/linux_x86_64/index.mdx

advocacy_docs/edb-postgres-ai/migration-etl/data-migration-service/terminology.mdx

advocacy_docs/edb-postgres-ai/migration-etl/index.mdx

install_template/config.yaml

install_template/templates/platformBase/_deploymentConstants.njk

install_template/templates/products/edb-data-migration-service-writer/ubuntu-20.04.njk

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

…vice/getting_started/index.mdx Co-authored-by: gvasquezvargas <[email protected]>

…ed mark as completed page

…d writer

… removed generated files no longer needed

yidian1997 · 2025-03-27T10:11:16Z

...etl/data-migration-service/getting_started/installing2/linux_arm64/edb-dms-writer_rhel_9.mdx

+## Prerequisites
+
+Before you begin the installation process:
+
+- Set up the EDB repository.
+
+  Setting up the repository is a one-time task. If you have already set up your repository, you don't need to perform this step.
+
+  To determine if your repository exists, enter this command:
+
+  `dnf repolist | grep enterprisedb`
+
+  If no output is generated, the repository isn't installed.
+
+  To set up the EDB repository:
+
+  1. Go to [EDB repositories](https://www.enterprisedb.com/repos-downloads).
+
+  1. Select the button that provides access to the EDB repository.
+
+  1. Select the platform and software that you want to download.
+
+  1. Follow the instructions for setting up the EDB repository.
+
+## Install the package
+
+Install the EDB DMS Writer (packaged as `cdcwriter`):
+
+```shell
+sudo dnf install cdcwriter
+```


Suggested change

## Prerequisites

Before you begin the installation process:

- Set up the EDB repository.

Setting up the repository is a one-time task. If you have already set up your repository, you don't need to perform this step.

To determine if your repository exists, enter this command:

`dnf repolist | grep enterprisedb`

If no output is generated, the repository isn't installed.

To set up the EDB repository:

1. Go to [EDB repositories](https://www.enterprisedb.com/repos-downloads).

1. Select the button that provides access to the EDB repository.

1. Select the platform and software that you want to download.

1. Follow the instructions for setting up the EDB repository.

## Install the package

Install the EDB DMS Writer (packaged as `cdcwriter`):

```shell

sudo dnf install cdcwriter

```

# Install using system package manager

## Prerequisites

Before you begin the installation process:

- Set up the EDB repository.

Setting up the repository is a one-time task. If you have already set up your repository, you don't need to perform this step.

To determine if your repository exists, enter this command:

`dnf repolist | grep enterprisedb`

If no output is generated, the repository isn't installed.

To set up the EDB repository:

1. Go to [EDB repositories](https://www.enterprisedb.com/repos-downloads).

1. Select the button that provides access to the EDB repository.

1. Select the platform and software that you want to download.

1. Follow the instructions for setting up the EDB repository.

## Install the package

Install the EDB DMS Writer (packaged as `cdcwriter`) according to the instruction in EDB repositories

Example:

```shell

sudo dnf install cdcwriter

# Install using EDB provided installer

## Download EDB software installer from browser

1. Go to [EDB repositories](https://www.enterprisedb.com/repos-downloads).

2. Select the button that provides access to the EDB repository.

3. Select the platform and software that you want to download.

4. Go to the `Enterprise` link under the `Download EDB software packages from your browser` section

5. Download the cdcwriter installer

## Install the package

Install the EDB DMS Writer (packaged as `cdcwriter`) using the installer.

Hi @gvasquezvargas After the investigation, I found we have two approaches to installing the reader and writer.

Using system package manager
Currently missing cdcreader and cdcwriter in the software list, pending fix.

Using the EDB-provided installer
Can find our installer in the list

So I propose such kind of change to let user can choose the way to download and install they want to use. What do your think?

cc @jperozoa

jpe442

LGTM

gvasquezvargas requested a review from a team as a code owner March 11, 2025 19:23

yidian1997 reviewed Mar 12, 2025

View reviewed changes

advocacy_docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/index.mdx Outdated Show resolved Hide resolved

gvasquezvargas added the deploy Add this label to a PR and it will automatically be deployed to netlify label Mar 12, 2025

github-actions bot temporarily deployed to pull request March 12, 2025 08:57 Inactive

gvasquezvargas commented Mar 12, 2025

View reviewed changes

github-actions bot temporarily deployed to pull request March 12, 2025 09:41 Inactive

github-actions bot temporarily deployed to pull request March 12, 2025 11:03 Inactive

github-actions bot temporarily deployed to pull request March 17, 2025 09:49 Inactive

github-actions bot temporarily deployed to pull request March 17, 2025 10:20 Inactive

github-actions bot temporarily deployed to pull request March 17, 2025 16:37 Inactive

gvasquezvargas requested review from mlewandowski-edb and jperozoa March 18, 2025 15:55

github-actions bot temporarily deployed to pull request March 19, 2025 15:17 Inactive

jperozoa approved these changes Mar 19, 2025

View reviewed changes

mlewandowski-edb reviewed Mar 20, 2025

View reviewed changes

gvasquezvargas commented Mar 24, 2025

View reviewed changes

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx Show resolved Hide resolved

github-actions bot temporarily deployed to pull request March 24, 2025 11:42 Inactive

github-actions bot temporarily deployed to pull request March 25, 2025 15:46 Inactive

mlewandowski-edb reviewed Mar 25, 2025

View reviewed changes

github-actions bot temporarily deployed to pull request March 25, 2025 21:15 Inactive

gvasquezvargas force-pushed the data_mig_service/march_release_branch branch from 4122759 to 54bbd59 Compare March 26, 2025 12:36

github-actions bot temporarily deployed to pull request March 26, 2025 12:48 Inactive

gvasquezvargas commented Mar 26, 2025

View reviewed changes

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx Outdated Show resolved Hide resolved

gvasquezvargas commented Mar 26, 2025

View reviewed changes

..._docs/edb-postgres-ai/migration-etl/data-migration-service/getting_started/config_writer.mdx Outdated Show resolved Hide resolved

github-actions bot temporarily deployed to pull request March 26, 2025 16:13 Inactive

github-actions bot temporarily deployed to pull request March 26, 2025 16:35 Inactive

gvasquezvargas changed the title ~~DMS: March release branch~~ [DMS] March release branch, release Thursday 27 Mar 27, 2025

gvasquezvargas and others added 2 commits March 27, 2025 10:13

DMS: March release branch

910235c

chore(ET-1872): add config_writer.mdx

03446eb

yidian1997 and others added 21 commits March 27, 2025 10:13

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

19a6240

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

00079dc

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

bbbd1b5

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

da9b04e

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

dd29add

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

2b17183

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

c942590

…vice/getting_started/config_writer.mdx Co-authored-by: gvasquezvargas <[email protected]>

Update advocacy_docs/edb-postgres-ai/migration-etl/data-migration-ser…

54af3ef

…vice/getting_started/index.mdx Co-authored-by: gvasquezvargas <[email protected]>

Initial code changes to support CDC Writer installation

a424492

Removed SLES 12 artifact

c939f91

Generated file changes needed to support CDC writer

a2a8969

Cross-linked DMS Writer installation to getting started

9d46c84

Rewording the rest of DMS documentation to include writer information

d720621

Updated terminology page with new terms, added Oracle 21c note, updat…

d550baa

…ed mark as completed page

Edited note on RW_SERVICE_HOST

c37e762

Code changes to generate install files for DMS reader and writer

c35b0da

Generated file changes for Ubuntu 24 install topics for DMS reader an…

1995809

…d writer

Change config.yaml to remove ubuntu 20 from dms reader and writer and…

4a341d6

… removed generated files no longer needed

Apply suggestions from Matt's review

b42db55

Removed ubuntu 20 file for DMS writer

58bc8e1

Reader/writer ID info

2e76fa0

gvasquezvargas force-pushed the data_mig_service/march_release_branch branch from 7be3f75 to 2e76fa0 Compare March 27, 2025 09:13

github-actions bot temporarily deployed to pull request March 27, 2025 09:17 Inactive

yidian1997 reviewed Mar 27, 2025

View reviewed changes

jpe442 approved these changes Mar 27, 2025

View reviewed changes

gvasquezvargas merged commit 7693cc2 into develop Mar 27, 2025
6 checks passed

gvasquezvargas deleted the data_mig_service/march_release_branch branch March 27, 2025 15:01

gvasquezvargas mentioned this pull request Mar 27, 2025

Release: 2025-03-27a #6648

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DMS] March release branch, release Thursday 27 #6603

[DMS] March release branch, release Thursday 27 #6603

gvasquezvargas commented Mar 11, 2025 •

edited

Loading

github-actions bot commented Mar 12, 2025 •

edited

Loading

gvasquezvargas left a comment

jperozoa left a comment

mlewandowski-edb left a comment

mlewandowski-edb Mar 19, 2025

gvasquezvargas Mar 24, 2025

mlewandowski-edb Mar 25, 2025

mlewandowski-edb Mar 19, 2025

yidian1997 Mar 25, 2025

gvasquezvargas Mar 25, 2025

mlewandowski-edb Mar 25, 2025

mlewandowski-edb Mar 19, 2025

mlewandowski-edb Mar 19, 2025

gvasquezvargas Mar 24, 2025

yidian1997 Mar 25, 2025

yidian1997 Mar 25, 2025 •

edited

Loading

mlewandowski-edb Mar 19, 2025

gvasquezvargas Mar 24, 2025

mlewandowski-edb left a comment

mlewandowski-edb Mar 25, 2025

yidian1997 Mar 27, 2025

yidian1997 Mar 27, 2025

yidian1997 Mar 27, 2025

jpe442 left a comment

[DMS] March release branch, release Thursday 27 #6603

[DMS] March release branch, release Thursday 27 #6603

Conversation

gvasquezvargas commented Mar 11, 2025 • edited Loading

What Changed?

github-actions bot commented Mar 12, 2025 • edited Loading

gvasquezvargas left a comment

Choose a reason for hiding this comment

jperozoa left a comment

Choose a reason for hiding this comment

mlewandowski-edb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yidian1997 Mar 25, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mlewandowski-edb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jpe442 left a comment

Choose a reason for hiding this comment

gvasquezvargas commented Mar 11, 2025 •

edited

Loading

github-actions bot commented Mar 12, 2025 •

edited

Loading

yidian1997 Mar 25, 2025 •

edited

Loading