Skip to content

Commit b1df75b

Browse files
hfhf
committed
Initial commit
0 parents  commit b1df75b

10 files changed

+978
-0
lines changed

‎.github/workflows/publish-release.yaml

+76
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
name: Publish Release
2+
3+
on:
4+
release:
5+
types:
6+
- published
7+
8+
jobs:
9+
publish:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/checkout@v2
13+
- run: |
14+
set -x
15+
16+
yarn --pure-lockfile
17+
18+
yarn build
19+
20+
rm -rf node_modules
21+
22+
yarn --prod --pure-lockfile
23+
24+
zip -9 -r lambda.zip \
25+
node_modules \
26+
index.js \
27+
index.js.map \
28+
index.ts \
29+
yarn.lock \
30+
package.json \
31+
tsconfig.json \
32+
README.md \
33+
template.yaml \
34+
LICENSE
35+
- uses: aws-actions/configure-aws-credentials@v1
36+
with:
37+
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
38+
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
39+
aws-region: us-east-1
40+
- env:
41+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
42+
run: |
43+
set -x
44+
45+
RELEASE_TYPE=""
46+
47+
if [ "true" == "${{ github.event.release.draft }}" ]
48+
then
49+
RELEASE_TYPE="draft-"
50+
fi
51+
52+
if [ "true" == "${{ github.event.release.prerelease }}" ]
53+
then
54+
RELEASE_TYPE="$(echo $RELEASE_TYPE)pre"
55+
fi
56+
57+
RELEASE_TYPE="$(echo $RELEASE_TYPE)release"
58+
59+
for region in us-east-1 eu-west-1
60+
do
61+
aws s3api put-object --region "$region" \
62+
--bucket "github-com-hf-pubpubsub-$region" \
63+
--key "$RELEASE_TYPE/cloudformation/${{ github.event.release.tag_name }}" \
64+
--content-type application/x-yaml \
65+
--body template.yaml
66+
67+
aws s3api put-object --region "$region" \
68+
--bucket "github-com-hf-pubpubsub-$region" \
69+
--key "$RELEASE_TYPE/lambda/${{ github.event.release.tag_name }}" \
70+
--content-type application/zip \
71+
--body lambda.zip
72+
done
73+
74+
gh release upload "${{ github.event.release.tag_name }}" \
75+
lambda.zip \
76+
template.yaml

‎.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
*.js
2+
*.js.map
3+
node_modules

‎.vimrc

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
let NERDTreeIgnore = ['.git$', '.cache$', 'node_modules$']
2+
let NERDTreeShowHidden = 1
3+
4+
:set backupcopy=yes

‎LICENSE

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Copyright 2021 Stojan Dimitrovski
2+
3+
Permission is hereby granted, free of charge, to any person obtaining a copy of
4+
this software and associated documentation files (the "Software"), to deal in
5+
the Software without restriction, including without limitation the rights to
6+
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
7+
of the Software, and to permit persons to whom the Software is furnished to do
8+
so, subject to the following conditions:
9+
10+
The above copyright notice and this permission notice shall be included in all
11+
copies or substantial portions of the Software.
12+
13+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
19+
SOFTWARE.

‎README.md

+78
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# PubPubSub
2+
3+
A ready-made AWS Lambda function you can invoke to publish messages into
4+
Google's PubSub system.
5+
6+
It's really useful for bridging AWS compute with Firebase or other Google Cloud
7+
projects.
8+
9+
## Setup
10+
11+
Follow the guide to setup [Workload Identity
12+
Federation][gcp-workload-identity-federation] in your Google Cloud projects.
13+
14+
Then you can deploy the code in this repository as a Node.js Lambda function in
15+
your AWS account.
16+
17+
Check out the Releases section for packages and instructions.
18+
19+
Hosted versions are available out of my personal AWS account, should you need
20+
to get running quickly. You're encouraged to host it on your own accounts and
21+
service is provided without any guarantees or warranty express or implied.
22+
23+
CloudFormation:
24+
25+
```
26+
https://github-com-hf-pubpubsub-<region>.s3.amazonaws.com/release/cloudformation/<release>
27+
```
28+
29+
Lambda:
30+
31+
```
32+
https://github-com-hf-pubpubsub-<region>.s3.amazonaws.com/release/lambda/<release>
33+
```
34+
35+
Currently available in: `us-east-1`, `eu-west-1`. Open an issue to
36+
request availability in other regions.
37+
38+
Note that you're only allowed to access the S3 object, not the bucket itself.
39+
You must do it using AWS credentials, and your identity will be visible in my
40+
CloudTrail account.
41+
42+
### Configuration
43+
44+
The Lambda function expects these environment variables:
45+
46+
- `GOOGLE_CLOUD_PROJECT_NUMBER` GCP project number where the Workload Identity
47+
Federation provider resides
48+
- `GOOGLE_CLOUD_WEB_IDENTITY_POOL` ID of the web identity pool
49+
- `GOOGLE_CLOUD_WEB_IDENTITY_POOL_AWS_PROVIDER` ID of the AWS provider in the
50+
web identity pool
51+
- `GOOGLE_CLOUD_SERVICE_ACCOUNT_EMAIL` Service account email to impersonate
52+
- `GOOGLE_CLOUD_DEFAULT_PROJECT_ID` (Optional) Project ID where messages
53+
without a `projectId` property will be directed to
54+
55+
## Use
56+
57+
The Lambda function can be invoked synchronously or asynchronously.
58+
59+
It accepts a JSON object with the following structure:
60+
61+
- `topic` Pub/Sub topic to which to publish
62+
- `message` A string representing the message to be published
63+
- `base64` (Optional) A boolean whether the `mesasge` is encoded in Base64
64+
- `projectId` (Optional) GCP project ID where the `topic` lives
65+
66+
It outputs a JSON object with the following structure:
67+
68+
- `topic` Pub/Sub topic where the message was published
69+
- `projectId` GCP project ID where the `topic` lives
70+
- `messageId` ID of the message in the `topic`
71+
72+
## License
73+
74+
Copyright &copy; 2022 Stojan Dimitrovski. Some rights reserved.
75+
76+
Licensed under the MIT X11 License. You can find a copy of it under `LICENSE`.
77+
78+
[gcp-workload-identity-federation]: https://cloud.google.com/iam/docs/configuring-workload-identity-federation#aws

‎index.ts

+97
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
import "source-map-support/register";
2+
import * as fs from "fs";
3+
import { PubSub } from "@google-cloud/pubsub";
4+
5+
const setupGoogleCloudCredentials = () => {
6+
const gcloudServiceAccountCredentials = {
7+
type: "external_account",
8+
subject_token_type: "urn:ietf:params:aws:token-type:aws4_request",
9+
credential_source: {
10+
environment_id: "aws1",
11+
region_url:
12+
"http://169.254.169.254/latest/meta-data/placement/availability-zone",
13+
url: "http://169.254.169.254/latest/meta-data/iam/security-credentials",
14+
regional_cred_verification_url:
15+
"https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
16+
},
17+
token_url: "https://sts.googleapis.com/v1/token",
18+
audience: `//iam.googleapis.com/projects/${process.env.GOOGLE_CLOUD_PROJECT_NUMBER}/locations/global/workloadIdentityPools/${process.env.GOOGLE_CLOUD_WEB_IDENTITY_POOL}/providers/${process.env.GOOGLE_CLOUD_WEB_IDENTITY_POOL_AWS_PROVIDER}`,
19+
service_account_impersonation_url: `https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${process.env.GOOGLE_CLOUD_SERVICE_ACCOUNT_EMAIL}:generateAccessToken`,
20+
};
21+
22+
process.env.GOOGLE_APPLICATION_CREDENTIALS =
23+
"/tmp/gcloud-service-account-credentials.json";
24+
25+
fs.writeFileSync(
26+
process.env.GOOGLE_APPLICATION_CREDENTIALS,
27+
JSON.stringify(gcloudServiceAccountCredentials)
28+
);
29+
30+
console.log(
31+
"Google Cloud Service Account Credentials",
32+
gcloudServiceAccountCredentials
33+
);
34+
};
35+
36+
const clients: { readonly [projectId: string]: PubSub } = (() => {
37+
const underlying: any = {};
38+
let credentialsSetup = false;
39+
40+
const projectIdRegexp = /^[a-z][a-z-0-9-]{4,28}[a-z0-9]$/;
41+
42+
return new Proxy(underlying, {
43+
get: (_, projectId) => {
44+
if ("string" !== typeof projectId) {
45+
return underlying[projectId];
46+
}
47+
48+
if (!projectIdRegexp.test(projectId)) {
49+
throw new Error(`Invalid project id '${projectId}'`);
50+
}
51+
52+
if (!credentialsSetup) {
53+
setupGoogleCloudCredentials();
54+
55+
credentialsSetup = true;
56+
}
57+
58+
if (!underlying[projectId]) {
59+
underlying[projectId] = new PubSub({ projectId });
60+
}
61+
62+
return underlying[projectId];
63+
},
64+
set: () => {
65+
throw new Error("Read only object!");
66+
},
67+
});
68+
})();
69+
70+
export const handler = async (event: {
71+
topic: string;
72+
message: string;
73+
projectId?: string;
74+
base64?: boolean;
75+
}) => {
76+
const projectId =
77+
event.projectId || process.env.GOOGLE_CLOUD_DEFAULT_PROJECT_ID || "";
78+
79+
try {
80+
const messageId = await clients[projectId]
81+
.topic(event.topic)
82+
.publish(Buffer.from(event.message, event.base64 ? "base64" : "utf-8"));
83+
84+
return {
85+
messageId,
86+
projectId,
87+
topic: event.topic,
88+
};
89+
} catch (e) {
90+
console.error(
91+
`Unable to publish message to topic '${event.topic}' on project '${projectId}'`,
92+
e
93+
);
94+
95+
throw e;
96+
}
97+
};

‎package.json

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"private": true,
3+
"license": "MIT",
4+
"author": "Stojan Dimitrovski <[email protected]>",
5+
"scripts": {
6+
"prettier": "prettier --write .",
7+
"build": "tsc"
8+
},
9+
"devDependencies": {
10+
"@tsconfig/node12": "^1.0.9",
11+
"prettier": "^2.5.1",
12+
"typescript": "^4.5.4"
13+
},
14+
"dependencies": {
15+
"@google-cloud/pubsub": "^2.18.4",
16+
"source-map-support": "^0.5.21"
17+
}
18+
}

‎template.yaml

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
Parameters:
2+
PPubPubSubCodeKey:
3+
Type: String
4+
PGoogleCloudProjectNumber:
5+
Type: String
6+
PGoogleCloudWebIdentityPool:
7+
Type: String
8+
PGoogleCloudWebIdentityPoolAWSProvider:
9+
Type: String
10+
PGoogleCloudServiceAccountEmail:
11+
Type: String
12+
PGoogleCloudDefaultProjectId:
13+
Type: String
14+
Default: ""
15+
16+
Resources:
17+
RoleLambdaPubPubSub:
18+
Type: AWS::IAM::Role
19+
Properties:
20+
AssumeRolePolicyDocument:
21+
Version: 2012-10-17
22+
Statement:
23+
- Effect: Allow
24+
Action: sts:AssumeRole
25+
Principal:
26+
Service: lambda.amazonaws.com
27+
ManagedPolicyArns:
28+
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
29+
LambdaPubPubSub:
30+
Type: AWS::Lambda::Function
31+
Properties:
32+
Runtime: nodejs14.x
33+
Role: !Sub ${RoleLambdaPubPubSub.Arn}
34+
Timeout: 60
35+
MemorySize: 192
36+
Handler: index.handler
37+
Code:
38+
S3Bucket: !Sub github-com-hf-pubpubsub-${AWS::Region}
39+
S3Key: !Sub ${PPubPubSubCodeKey}
40+
Environment:
41+
Variables:
42+
GOOGLE_CLOUD_PROJECT_NUMBER: !Ref PGoogleCloudProjectNumber
43+
GOOGLE_CLOUD_WEB_IDENTITY_POOL: !Ref PGoogleCloudWebIdentityPool
44+
GOOGLE_CLOUD_WEB_IDENTITY_POOL_AWS_PROVIDER: !Ref PGoogleCloudWebIdentityPoolAWSProvider
45+
GOOGLE_CLOUD_SERVICE_ACCOUNT_EMAIL: !Ref PGoogleCloudServiceAccountEmail
46+
GOOGLE_CLOUD_DEFAULT_PROJECT_ID: !Ref PGoogleCloudDefaultProjectId

‎tsconfig.json

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"extends": "@tsconfig/node12/tsconfig.json",
3+
"compilerOptions": {
4+
"sourceMap": true
5+
},
6+
"include": ["*.ts"]
7+
}

0 commit comments

Comments
 (0)