Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL instances / databases #18

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
da9a92f
WIP: start working on fetching information regarding SQL instances fo…
christeredvartsen Mar 14, 2024
c7ff9bb
feat(sql): add status to sqlinstance
tommytroen Mar 14, 2024
3269e92
update condition on an sqlinstance to mark it as failing
christeredvartsen Mar 14, 2024
5ac29bc
move sqlinstances into TeamStatus for now
christeredvartsen Mar 14, 2024
75b6c63
add query to fetch SQL instances
christeredvartsen Mar 18, 2024
2d5caa5
feat(sql): add env type to sqlinstances
tommytroen Mar 18, 2024
9b75dda
feat(sql): return connectionname and ishealthy
tommytroen Mar 18, 2024
1cc17b7
add resolvers for app and team
christeredvartsen Mar 18, 2024
659e56e
wip: cost calculation for SQL instances
christeredvartsen Mar 18, 2024
76ce965
wip(sql): first take at metrics from google
tommytroen Mar 19, 2024
2dfb319
add(graphql): metrics resolver for cpu utlization
ybelMekk Mar 19, 2024
a96ad85
add high availability info
christeredvartsen Mar 20, 2024
a31180f
add memory utilization
christeredvartsen Mar 20, 2024
0cda171
fetch disk utilization for SQL instances
christeredvartsen Mar 20, 2024
2522ff8
feat: add quota and cores to utilization
tommytroen Mar 20, 2024
aaffeeb
refactor: remove switch for metric type
tommytroen Mar 20, 2024
c632c04
update(sql): with more data from sqlinstance
ybelMekk Mar 22, 2024
0e77f4e
create root query for SQL instance
christeredvartsen Mar 22, 2024
5cf1abe
feat: add projectId to sqlinstance
tommytroen Mar 22, 2024
5b60d26
add SQL database informer so we can list databases
christeredvartsen Mar 25, 2024
29a52ff
add resolver for team
christeredvartsen Mar 25, 2024
5c46f0d
refactor: remove diskSize from sqlinstance type
tommytroen Mar 25, 2024
b7f52b7
make generate after rebase
christeredvartsen Mar 25, 2024
921f138
fix: add nil checks on backupconfiguration
tommytroen Mar 26, 2024
306bcd0
feat: add logger to metrics for sqlinstance
tommytroen Mar 26, 2024
35cedef
remove unused func
christeredvartsen Apr 2, 2024
66f0f9d
create separate resolvers for the metric classes
christeredvartsen Apr 3, 2024
fb9f280
feat: extract more info from sqlinstance
tommytroen Apr 3, 2024
1507d00
add(postgres): maintenance version
ybelMekk Apr 3, 2024
ba19098
fix(postgres): sql backup
ybelMekk Apr 3, 2024
3059f49
add(postgres): gql ID for App storage resolver
ybelMekk Apr 4, 2024
a956988
refactor: remove unused fields from sqlinstance schema
tommytroen Apr 4, 2024
ee83111
refactor: fix todos and validate date intervals
tommytroen Apr 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[*.graphqls]
indent_size = 2
indent_style = space
4 changes: 2 additions & 2 deletions data/k8s/dev/devteam/app-w-all-storage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ spec:
sqlInstances:
- type: POSTGRES_15
cascadingDelete: true
name: deleteme
- type: POSTGRES_15
name: contests
- type: POSTGRES_14
name: donotdelete
bigQueryDatasets:
- name: deleteme_bq
Expand Down
49 changes: 49 additions & 0 deletions data/k8s/dev/devteam/sqldatabase.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLDatabase
metadata:
annotations:
cnrm.cloud.google.com/deletion-policy: abandon
cnrm.cloud.google.com/management-conflict-prevention-policy: none
cnrm.cloud.google.com/project-id: nais-dev-2e7b
cnrm.cloud.google.com/state-into-spec: merge
nais.io/deploymentCorrelationID: c28520a3-4232-4ece-ae48-330deb16c853
labels:
app: contests
app.kubernetes.io/instance: contests
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: contests
helm.sh/chart: contests-2023.1.12-172b2e7
name: contests
spec:
charset: UTF8
collation: en_US.UTF8
deletionPolicy: DELETE
instanceRef:
name: contests
resourceID: contests
---
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLDatabase
metadata:
annotations:
cnrm.cloud.google.com/deletion-policy: abandon
cnrm.cloud.google.com/management-conflict-prevention-policy: none
cnrm.cloud.google.com/project-id: nais-dev-2e7b
cnrm.cloud.google.com/state-into-spec: merge
nais.io/deploymentCorrelationID: 1033f984-ed8b-4906-9d1b-4ebbda1d42b0
labels:
app: some-app
app.kubernetes.io/instance: some-app
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: some-app
app.kubernetes.io/version: 1.16.0
helm.sh/chart: some-app-0.1.1-20230825095400
team: nais
name: some-other-database
spec:
charset: UTF8
collation: en_US.UTF8
deletionPolicy: DELETE
instanceRef:
name: contests
resourceID: some-other-database
153 changes: 153 additions & 0 deletions data/k8s/dev/devteam/sqlinstance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLInstance
metadata:
annotations:
cnrm.cloud.google.com/deletion-policy: abandon
cnrm.cloud.google.com/management-conflict-prevention-policy: none
cnrm.cloud.google.com/mutable-but-unreadable-fields: '{}'
cnrm.cloud.google.com/observed-secret-versions: '{}'
cnrm.cloud.google.com/project-id: nais-dev-2e7b
cnrm.cloud.google.com/state-into-spec: merge
nais.io/deploymentCorrelationID: c53778c4-f03f-41a5-ae2f-0ca6a833019e
labels:
app: app-w-all-storage
app.kubernetes.io/instance: app-w-all-storage
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: app-w-all-storage
helm.sh/chart: app-w-all-storage-2024-02-22-115844-b411609
team: nais
name: contests
namespace: nais-system
spec:
databaseVersion: POSTGRES_15
instanceType: CLOUD_SQL_INSTANCE
maintenanceVersion: POSTGRES_15_10.R20240130.00_05
region: europe-north1
resourceID: contests
settings:
activationPolicy: ALWAYS
availabilityType: ZONAL
backupConfiguration:
backupRetentionSettings:
retainedBackups: 7
retentionUnit: COUNT
enabled: true
pointInTimeRecoveryEnabled: false
startTime: "02:00"
transactionLogRetentionDays: 7
connectorEnforcement: NOT_REQUIRED
databaseFlags:
- name: cloudsql.iam_authentication
value: "on"
diskAutoresize: false
diskAutoresizeLimit: 0
diskSize: 10
diskType: PD_SSD
insightsConfig:
queryInsightsEnabled: true
queryStringLength: 1024
recordApplicationTags: false
recordClientAddress: false
ipConfiguration:
ipv4Enabled: true
requireSsl: true
locationPreference:
zone: europe-north1-a
pricingPlan: PER_USE
tier: db-f1-micro
status:
conditions:
- lastTransitionTime: "2024-01-23T09:30:20Z"
message: The resource is up to date
reason: UpToDate
status: "True"
type: Ready
connectionName: nais-dev-123:europe-north1:app-w-all-storage
firstIpAddress: 1.2.3.4
instanceType: CLOUD_SQL_INSTANCE
ipAddress:
- ipAddress: 1.2.3.4
type: PRIMARY
- ipAddress: 1.2.3.4
type: OUTGOING
observedGeneration: 104
publicIpAddress: 1.2.3.4
selfLink: https://sqladmin.googleapis.com/sql/v1beta4/projects/nais-dev-123/instances/app-w-all-storage
serviceAccountEmailAddress: [email protected]
---
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLInstance
metadata:
annotations:
cnrm.cloud.google.com/deletion-policy: abandon
cnrm.cloud.google.com/management-conflict-prevention-policy: none
cnrm.cloud.google.com/mutable-but-unreadable-fields: '{}'
cnrm.cloud.google.com/observed-secret-versions: '{}'
cnrm.cloud.google.com/project-id: nais-dev-2e7b
cnrm.cloud.google.com/state-into-spec: merge
nais.io/deploymentCorrelationID: c53778c4-f03f-41a5-ae2f-0ca6a833019e
labels:
app: app-w-all-storage
app.kubernetes.io/instance: app-w-all-storage
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: app-w-all-storage
helm.sh/chart: app-w-all-storage-2024-02-22-115844-b411609
team: nais
name: donotdelete
namespace: nais-system
spec:
databaseVersion: POSTGRES_14
instanceType: CLOUD_SQL_INSTANCE
maintenanceVersion: POSTGRES_14_10.R20240130.00_05
region: europe-north1
resourceID: donotdelete
settings:
activationPolicy: ALWAYS
availabilityType: ZONAL
backupConfiguration:
backupRetentionSettings:
retainedBackups: 7
retentionUnit: COUNT
enabled: true
pointInTimeRecoveryEnabled: false
startTime: "02:00"
transactionLogRetentionDays: 7
connectorEnforcement: NOT_REQUIRED
databaseFlags:
- name: cloudsql.iam_authentication
value: "on"
diskAutoresize: false
diskAutoresizeLimit: 0
diskSize: 10
diskType: PD_SSD
insightsConfig:
queryInsightsEnabled: true
queryStringLength: 1024
recordApplicationTags: false
recordClientAddress: false
ipConfiguration:
ipv4Enabled: true
requireSsl: true
locationPreference:
zone: europe-north1-a
pricingPlan: PER_USE
tier: db-f1-micro
status:
conditions:
- lastTransitionTime: "2024-01-23T09:30:20Z"
message: "Update call failed: error applying desired state: summary: Error, failed to update instance settings for : googleapi: Error 400: Invalid request: backup retention should be >= transaction log retention.,"
reason: UpdateFailed
status: "False"
type: Ready
connectionName: nais-dev-123:europe-north1:app-w-all-storage
firstIpAddress: 1.2.3.4
instanceType: CLOUD_SQL_INSTANCE
ipAddress:
- ipAddress: 1.2.3.4
type: PRIMARY
- ipAddress: 1.2.3.4
type: OUTGOING
observedGeneration: 104
publicIpAddress: 1.2.3.4
selfLink: https://sqladmin.googleapis.com/sql/v1beta4/projects/nais-dev-123/instances/app-w-all-storage
serviceAccountEmailAddress: [email protected]
76 changes: 76 additions & 0 deletions data/k8s/dev/nais/sqlinstance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLInstance
metadata:
annotations:
cnrm.cloud.google.com/deletion-policy: abandon
cnrm.cloud.google.com/management-conflict-prevention-policy: none
cnrm.cloud.google.com/mutable-but-unreadable-fields: '{}'
cnrm.cloud.google.com/observed-secret-versions: '{}'
cnrm.cloud.google.com/project-id: nais-dev-2e7b
cnrm.cloud.google.com/state-into-spec: merge
nais.io/deploymentCorrelationID: c53778c4-f03f-41a5-ae2f-0ca6a833019e
labels:
app: contests
app.kubernetes.io/instance: contests
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: contests
helm.sh/chart: contests-2024-02-22-115844-b411609
team: nais
name: contests
namespace: nais-system
spec:
databaseVersion: POSTGRES_14
instanceType: CLOUD_SQL_INSTANCE
maintenanceVersion: POSTGRES_14_10.R20240130.00_05
region: europe-north1
resourceID: contests
settings:
activationPolicy: ALWAYS
availabilityType: ZONAL
backupConfiguration:
backupRetentionSettings:
retainedBackups: 7
retentionUnit: COUNT
enabled: true
pointInTimeRecoveryEnabled: false
startTime: "02:00"
transactionLogRetentionDays: 7
connectorEnforcement: NOT_REQUIRED
databaseFlags:
- name: cloudsql.iam_authentication
value: "on"
diskAutoresize: false
diskAutoresizeLimit: 0
diskSize: 10
diskType: PD_SSD
insightsConfig:
queryInsightsEnabled: true
queryStringLength: 1024
recordApplicationTags: false
recordClientAddress: false
ipConfiguration:
ipv4Enabled: true
requireSsl: true
locationPreference:
zone: europe-north1-a
pricingPlan: PER_USE
tier: db-f1-micro
status:
conditions:
- lastTransitionTime: "2024-01-23T09:30:20Z"
message: The resource is up to date
reason: UpToDate
status: "True"
type: Ready
connectionName: nais-dev-123:europe-north1:contests
firstIpAddress: 1.2.3.4
instanceType: CLOUD_SQL_INSTANCE
ipAddress:
- ipAddress: 1.2.3.4
type: PRIMARY
- ipAddress: 1.2.3.4
type: OUTGOING
observedGeneration: 104
publicIpAddress: 1.2.3.4
selfLink: https://sqladmin.googleapis.com/sql/v1beta4/projects/nais-dev-123/instances/contests
serviceAccountEmailAddress: [email protected]
79 changes: 79 additions & 0 deletions data/k8s/superprod/devteam/sqlinstance.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLInstance
metadata:
annotations:
cnrm.cloud.google.com/deletion-policy: abandon
cnrm.cloud.google.com/management-conflict-prevention-policy: none
cnrm.cloud.google.com/mutable-but-unreadable-fields: '{}'
cnrm.cloud.google.com/observed-secret-versions: '{}'
cnrm.cloud.google.com/project-id: nais-dev-2e7b
cnrm.cloud.google.com/state-into-spec: merge
nais.io/deploymentCorrelationID: c53778c4-f03f-41a5-ae2f-0ca6a833019e
labels:
app: contests
app.kubernetes.io/instance: contests
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: contests
helm.sh/chart: contests-2024-02-22-115844-b411609
team: nais
name: contests
namespace: nais-system
spec:
databaseVersion: POSTGRES_14
instanceType: CLOUD_SQL_INSTANCE
maintenanceVersion: POSTGRES_14_10.R20240130.00_05
region: europe-north1
resourceID: contests
settings:
activationPolicy: ALWAYS
availabilityType: REGIONAL
backupConfiguration:
backupRetentionSettings:
retainedBackups: 7
retentionUnit: COUNT
enabled: true
pointInTimeRecoveryEnabled: false
startTime: "02:00"
transactionLogRetentionDays: 7
connectorEnforcement: NOT_REQUIRED
databaseFlags:
- name: cloudsql.iam_authentication
value: "on"
diskAutoresize: false
diskAutoresizeLimit: 0
diskSize: 10
diskType: PD_SSD
insightsConfig:
queryInsightsEnabled: true
queryStringLength: 1024
recordApplicationTags: false
recordClientAddress: false
ipConfiguration:
ipv4Enabled: true
requireSsl: true
locationPreference:
zone: europe-north1-a
pricingPlan: PER_USE
maintenanceWindow:
day: 1
hour: 4
tier: db-f1-micro
status:
conditions:
- lastTransitionTime: "2024-01-23T09:30:20Z"
message: The resource is up to date
reason: UpToDate
status: "True"
type: Ready
connectionName: nais-dev-123:europe-north1:contests
firstIpAddress: 1.2.3.4
instanceType: CLOUD_SQL_INSTANCE
ipAddress:
- ipAddress: 1.2.3.4
type: PRIMARY
- ipAddress: 1.2.3.4
type: OUTGOING
observedGeneration: 104
publicIpAddress: 1.2.3.4
selfLink: https://sqladmin.googleapis.com/sql/v1beta4/projects/nais-dev-123/instances/contests
serviceAccountEmailAddress: [email protected]
Loading