feat: [PIPE-25660]: Handling for pipeline hook

[shivamnegi94]

Sample Payload for Pipeline Gitlab Event:
{ "object_kind": "pipeline", "object_attributes":{ "id": 31, "iid": 3, "name": "Pipeline for branch: master", "ref": "master", "tag": false, "sha": "bcbb5ec396a2c0f828686f14fac9b80b780504f2", "before_sha": "bcbb5ec396a2c0f828686f14fac9b80b780504f2", "source": "merge_request_event", "status": "success", "stages":[ "build", "test", "deploy" ], "created_at": "2016-08-12 15:23:28 UTC", "finished_at": "2016-08-12 15:26:29 UTC", "duration": 63, "variables": [ { "key": "NESTOR_PROD_ENVIRONMENT", "value": "us-west-1" } ], "url": "http://example.com/gitlab-org/gitlab-test/-/pipelines/31" }, "merge_request": { "id": 1, "iid": 1, "title": "Test", "source_branch": "test", "source_project_id": 1, "target_branch": "master", "target_project_id": 1, "state": "opened", "merge_status": "can_be_merged", "detailed_merge_status": "mergeable", "url": "http://192.168.64.1:3005/gitlab-org/gitlab-test/merge_requests/1" }, "user":{ "id": 1, "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon", "email": "[email protected]" }, "project":{ "id": 1, "name": "Gitlab Test", "description": "Atque in sunt eos similique dolores voluptatem.", "web_url": "http://192.168.64.1:3005/gitlab-org/gitlab-test", "avatar_url": null, "git_ssh_url": "[email protected]:gitlab-org/gitlab-test.git", "git_http_url": "http://192.168.64.1:3005/gitlab-org/gitlab-test.git", "namespace": "Gitlab Org", "visibility_level": 20, "path_with_namespace": "gitlab-org/gitlab-test", "default_branch": "master" }, "commit":{ "id": "bcbb5ec396a2c0f828686f14fac9b80b780504f2", "message": "test\n", "timestamp": "2016-08-12T17:23:21+02:00", "url": "http://example.com/gitlab-org/gitlab-test/commit/bcbb5ec396a2c0f828686f14fac9b80b780504f2", "author":{ "name": "User", "email": "[email protected]" } }, "source_pipeline":{ "project":{ "id": 41, "web_url": "https://gitlab.example.com/gitlab-org/upstream-project", "path_with_namespace": "gitlab-org/upstream-project" }, "pipeline_id": 30, "job_id": 3401 }, "builds":[ { "id": 380, "stage": "deploy", "name": "production", "status": "skipped", "created_at": "2016-08-12 15:23:28 UTC", "started_at": null, "finished_at": null, "duration": null, "queued_duration": null, "failure_reason": null, "when": "manual", "manual": true, "allow_failure": false, "user":{ "id": 1, "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon", "email": "[email protected]" }, "runner": null, "artifacts_file":{ "filename": null, "size": null }, "environment": { "name": "production", "action": "start", "deployment_tier": "production" } }, { "id": 377, "stage": "test", "name": "test-image", "status": "success", "created_at": "2016-08-12 15:23:28 UTC", "started_at": "2016-08-12 15:26:12 UTC", "finished_at": "2016-08-12 15:26:29 UTC", "duration": 17.0, "queued_duration": 196.0, "failure_reason": null, "when": "on_success", "manual": false, "allow_failure": false, "user":{ "id": 1, "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon", "email": "[email protected]" }, "runner": { "id": 380987, "description": "shared-runners-manager-6.gitlab.com", "active": true, "runner_type": "instance_type", "is_shared": true, "tags": [ "linux", "docker", "shared-runner" ] }, "artifacts_file":{ "filename": null, "size": null }, "environment": null }, { "id": 378, "stage": "test", "name": "test-build", "status": "failed", "created_at": "2016-08-12 15:23:28 UTC", "started_at": "2016-08-12 15:26:12 UTC", "finished_at": "2016-08-12 15:26:29 UTC", "duration": 17.0, "queued_duration": 196.0, "failure_reason": "script_failure", "when": "on_success", "manual": false, "allow_failure": false, "user":{ "id": 1, "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon", "email": "[email protected]" }, "runner": { "id":380987, "description":"shared-runners-manager-6.gitlab.com", "active":true, "runner_type": "instance_type", "is_shared": true, "tags": [ "linux", "docker" ] }, "artifacts_file":{ "filename": null, "size": null }, "environment": null }, { "id": 376, "stage": "build", "name": "build-image", "status": "success", "created_at": "2016-08-12 15:23:28 UTC", "started_at": "2016-08-12 15:24:56 UTC", "finished_at": "2016-08-12 15:25:26 UTC", "duration": 17.0, "queued_duration": 196.0, "failure_reason": null, "when": "on_success", "manual": false, "allow_failure": false, "user":{ "id": 1, "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon", "email": "[email protected]" }, "runner": { "id": 380987, "description": "shared-runners-manager-6.gitlab.com", "active": true, "runner_type": "instance_type", "is_shared": true, "tags": [ "linux", "docker" ] }, "artifacts_file":{ "filename": null, "size": null }, "environment": null }, { "id": 379, "stage": "deploy", "name": "staging", "status": "created", "created_at": "2016-08-12 15:23:28 UTC", "started_at": null, "finished_at": null, "duration": null, "queued_duration": null, "failure_reason": null, "when": "on_success", "manual": false, "allow_failure": false, "user":{ "id": 1, "name": "Administrator", "username": "root", "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon", "email": "[email protected]" }, "runner": null, "artifacts_file":{ "filename": null, "size": null }, "environment": { "name": "staging", "action": "start", "deployment_tier": "staging" } } ] }

[bradrydzewski]

two comments

1. add unit test / sample payloads
2. add support for github or gitlab. the abstraction looks good at first glance, but we cannot know for sure until we implement a second provider

[bradrydzewski]

@abhinav-harness @johannesHarness can you review? The question I have is whether this belongs in the go-scm abstraction library of if this belongs in a vendor-specific Go library. The purpose of this repository is to abstract common functionality across all git providers (github, bitbucket, gitlab, etc). I'm not sure if pipelines and pipeline executions can be cleanly abstracted across vendors. If yes, we can include in the library. If no, folks should use vendor specific libraries to access this data (e.g. go-github, go-gitlab, etc). At the moment the abstraction looks very specific to Drone / Gitness.

[bradrydzewski]

I think the harness-go-sdk is probably a better home for this functionality
https://github.com/harness/harness-go-sdk

The go-scm library is meant to abstract common functionality across providers. I'm not sure this will be possible for Pipelines given the significant differences across vendors.

[abhinav-harness]

@shivamnegi94 there are code comments - but other than that please check if you can add more providers - if not this code has to go in scm-service. CI does put a lot of code like this in there

[bradrydzewski]

do we want to store run_number (GitHub) / object_attributes.id (GitLab) as Execution.Number and then remove Execution.ID? Similar to PullRequest the auto-incrementing number may be more useful to the consumer of the hook than the database unique id.

for bitbucket, I didn't see the number in the payload (maybe I missed it) but looks like it could be extracted from the url ... ex extract 4 from https://bitbucket.org/.../home#!/results/4

[shivamnegi94]

do we want to store run_number (GitHub) / object_attributes.id (GitLab) as Execution.Number and then remove Execution.ID? Similar to PullRequest the auto-incrementing number may be more useful to the consumer of the hook than the database unique id.

for bitbucket, I didn't see the number in the payload (maybe I missed it) but looks like it could be extracted from the url ... ex extract 4 from https://bitbucket.org/.../home#!/results/4

I have addressed this as well in my latest changes