feat: add data import/export commands

[yann-soubeyrand]

This simplifies the provisioning of local development environments. One can setup its organization, project, users, etc, and export instance data to a JSON file. Then, when creating a new development environment, the JSON file can be imported to the newly created instance.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.97%. Comparing base (f3faabd) to head (5a7be1f).
Report is 54 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #156      +/-   ##
==========================================
+ Coverage   91.79%   91.97%   +0.18%     
==========================================
  Files           4        4              
  Lines         195      162      -33     
==========================================
- Hits          179      149      -30     
+ Misses         13       10       -3     
  Partials        3        3              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[muhlemmer]

Thanks for the PR! This is a great contribution. I've left a few comments that will need to be resolved.

[muhlemmer]

Using the exported json file, the import fails with:

go run main.go data import --api=localhost:8080 --insecure --issuer=http://localhost:8080 --key=275218352408933106.json --data=export.json
2024/07/09 17:29:23 failed to unmarshal data: proto: (line 1:2): unknown field "orgs"
exit status 1

That's because the Request has a Oneof field that has to be populated first. I'll make a suggestion below in the file.

[muhlemmer]

Use the propper message type for unmarshall like this:

Suggested change

	var req pb.ImportDataRequest
	err = protojson.Unmarshal(data, &req)
	if err != nil {
	log.Fatalln("failed to unmarshal data:", err)
	return
	}
	resp, err := client.ImportData(context.Background(), &req)
	var reqOrgs pb.ImportDataOrg
	err = protojson.Unmarshal(data, &reqOrgs)
	if err != nil {
	log.Fatalln("failed to unmarshal data:", err)
	return
	}
	resp, err := client.ImportData(context.Background(), &pb.ImportDataRequest{
	Data: &pb.ImportDataRequest_DataOrgs{
	DataOrgs: &reqOrgs,
	},
	Timeout: "5m", // TODO: from command line argument!
	})

Timeout is a required field and will need to be parsed from the command line arguments as well.

[muhlemmer]

I feel that these 3 command line options are quite redundant.

For example:

1. If the issuer is http://localhost:8080, you already know the API endpoint is localhost:8080 and the connection is insecure.
2. If the issuer is https://<instance>.zitadel.cloud, you already know the API endpoint is <instance>.zitadel.cloud:443 and the connection is secure.

For this example you'll need to parse the URL of the issuer and imply the the API configuration from there.

The reverse can also be applied:

1. A --host flag can point to the API and will become part of the issuer
2. The --insecure flag toggles https to http in front of the issuer. The API port toggles between 443 and 80 by default.
3. A --port flag sets / overrides the port suffix to the host for the issuer and API.

In the last example, if one would only set <instance>.zitadel.cloud, the default will create issuer https://<instance>.zitadel.cloud and API endpoint <instance>.zitadel.cloud:443.

For user friendlyness I would want to stick to one of the above solutions, you may pick one.

[muhlemmer]

Hi @yann-soubeyrand, do you intend to continue work on this PR?

[yann-soubeyrand]

Hi @muhlemmer, sorry for the late reply. Yes, I’d be glad to work on this again, but I’ve just no time for it for the moment. My first intention when I opened this PR was to share the code which I used in a project.