Skip to content

Commit 14521e4

Browse files
kron4egkron4eg
committed
GCE dualstack support
This PR introduces a dedicated terraform config for GCE with IPv6 enabled Signed-off-by: Artiom Diomin <[email protected]>
1 parent 8b8d794 commit 14521e4

14 files changed

+647
-12
lines changed

‎examples/terraform/gce-ipv6/README.md

+94
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# GCE Quickstart Terraform configs
2+
3+
The GCE Quickstart Terraform configs can be used to create the needed
4+
infrastructure for a Kubernetes HA cluster. Check out the following
5+
[Creating Infrastructure guide][docs-infrastructure] to learn more about how to
6+
use the configs and how to provision a Kubernetes cluster using KubeOne.
7+
8+
[docs-infrastructure]: https://docs.kubermatic.com/kubeone/v1.9/guides/using-terraform-configs/
9+
10+
## GCE Provider configuration
11+
12+
### Credentials
13+
14+
Per <https://www.terraform.io/docs/providers/google/provider_reference.html#configuration-reference>
15+
either of the following ENV variables should be accessible:
16+
17+
* `GOOGLE_CREDENTIALS`
18+
* `GOOGLE_CLOUD_KEYFILE_JSON`
19+
* `GCLOUD_KEYFILE_JSON`
20+
21+
## Requirements
22+
23+
| Name | Version |
24+
|------|---------|
25+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0.0 |
26+
| <a name="requirement_google"></a> [google](#requirement\_google) | ~> 6 |
27+
28+
## Providers
29+
30+
| Name | Version |
31+
|------|---------|
32+
| <a name="provider_google"></a> [google](#provider\_google) | ~> 6 |
33+
34+
## Modules
35+
36+
No modules.
37+
38+
## Resources
39+
40+
| Name | Type |
41+
|------|------|
42+
| [google_compute_address.lb_ip](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_address) | resource |
43+
| [google_compute_firewall.common](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
44+
| [google_compute_firewall.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
45+
| [google_compute_firewall.internal](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
46+
| [google_compute_firewall.nodeports](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
47+
| [google_compute_forwarding_rule.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_forwarding_rule) | resource |
48+
| [google_compute_http_health_check.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_http_health_check) | resource |
49+
| [google_compute_instance.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance) | resource |
50+
| [google_compute_target_pool.control_plane_pool](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_target_pool) | resource |
51+
| [google_compute_image.control_plane_image](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_image) | data source |
52+
| [google_compute_network.network](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_network) | data source |
53+
| [google_compute_subnetwork.subnet](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_subnetwork) | data source |
54+
| [google_compute_zones.available](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_zones) | data source |
55+
56+
## Inputs
57+
58+
| Name | Description | Type | Default | Required |
59+
|------|-------------|------|---------|:--------:|
60+
| <a name="input_apiserver_alternative_names"></a> [apiserver\_alternative\_names](#input\_apiserver\_alternative\_names) | subject alternative names for the API Server signing cert. | `list(string)` | `[]` | no |
61+
| <a name="input_bastion_host_key"></a> [bastion\_host\_key](#input\_bastion\_host\_key) | Bastion SSH host public key | `string` | `null` | no |
62+
| <a name="input_cluster_autoscaler_max_replicas"></a> [cluster\_autoscaler\_max\_replicas](#input\_cluster\_autoscaler\_max\_replicas) | maximum number of replicas per MachineDeployment (requires cluster-autoscaler) | `number` | `0` | no |
63+
| <a name="input_cluster_autoscaler_min_replicas"></a> [cluster\_autoscaler\_min\_replicas](#input\_cluster\_autoscaler\_min\_replicas) | minimum number of replicas per MachineDeployment (requires cluster-autoscaler) | `number` | `0` | no |
64+
| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | Name of the cluster | `string` | n/a | yes |
65+
| <a name="input_control_plane_image_family"></a> [control\_plane\_image\_family](#input\_control\_plane\_image\_family) | Image family to use for provisioning instances | `string` | `"ubuntu-2404-lts-amd64"` | no |
66+
| <a name="input_control_plane_image_project"></a> [control\_plane\_image\_project](#input\_control\_plane\_image\_project) | Project of the image to use for provisioning instances | `string` | `"ubuntu-os-cloud"` | no |
67+
| <a name="input_control_plane_target_pool_members_count"></a> [control\_plane\_target\_pool\_members\_count](#input\_control\_plane\_target\_pool\_members\_count) | n/a | `number` | `3` | no |
68+
| <a name="input_control_plane_type"></a> [control\_plane\_type](#input\_control\_plane\_type) | GCE instance type | `string` | `"n1-standard-2"` | no |
69+
| <a name="input_control_plane_vm_count"></a> [control\_plane\_vm\_count](#input\_control\_plane\_vm\_count) | number of control plane instances | `number` | `3` | no |
70+
| <a name="input_control_plane_volume_size"></a> [control\_plane\_volume\_size](#input\_control\_plane\_volume\_size) | Size of the boot volume, in GB | `number` | `100` | no |
71+
| <a name="input_disable_kubeapi_loadbalancer"></a> [disable\_kubeapi\_loadbalancer](#input\_disable\_kubeapi\_loadbalancer) | E2E tests specific variable to disable usage of any loadbalancer in front of kubeapi-server | `bool` | `false` | no |
72+
| <a name="input_google_compute_network"></a> [google\_compute\_network](#input\_google\_compute\_network) | if given, will be used as a network | `string` | `"default"` | no |
73+
| <a name="input_google_compute_subnetwork"></a> [google\_compute\_subnetwork](#input\_google\_compute\_subnetwork) | if given, will be used as a subnet | `string` | `"default"` | no |
74+
| <a name="input_initial_machinedeployment_operating_system_profile"></a> [initial\_machinedeployment\_operating\_system\_profile](#input\_initial\_machinedeployment\_operating\_system\_profile) | Name of operating system profile for MachineDeployments, only applicable if operating-system-manager addon is enabled.<br>If not specified, the default value will be added by machine-controller addon. | `string` | `""` | no |
75+
| <a name="input_initial_machinedeployment_replicas"></a> [initial\_machinedeployment\_replicas](#input\_initial\_machinedeployment\_replicas) | Number of replicas per MachineDeployment | `number` | `2` | no |
76+
| <a name="input_network_tier"></a> [network\_tier](#input\_network\_tier) | The service-level to be provided for IPv6 traffic when the subnet has an external subnet.<br> Only PREMIUM or STANDARD tier is valid for IPv6. | `string` | `"PREMIUM"` | no |
77+
| <a name="input_project"></a> [project](#input\_project) | Project to be used for all resources | `string` | n/a | yes |
78+
| <a name="input_region"></a> [region](#input\_region) | GCP region to speak to | `string` | `"europe-west3"` | no |
79+
| <a name="input_ssh_agent_socket"></a> [ssh\_agent\_socket](#input\_ssh\_agent\_socket) | SSH Agent socket, default to grab from $SSH\_AUTH\_SOCK | `string` | `"env:SSH_AUTH_SOCK"` | no |
80+
| <a name="input_ssh_hosts_keys"></a> [ssh\_hosts\_keys](#input\_ssh\_hosts\_keys) | A list of SSH hosts public keys to verify | `list(string)` | `null` | no |
81+
| <a name="input_ssh_port"></a> [ssh\_port](#input\_ssh\_port) | SSH port to be used to provision instances | `number` | `22` | no |
82+
| <a name="input_ssh_private_key_file"></a> [ssh\_private\_key\_file](#input\_ssh\_private\_key\_file) | SSH private key file used to access instances | `string` | `""` | no |
83+
| <a name="input_ssh_public_key_file"></a> [ssh\_public\_key\_file](#input\_ssh\_public\_key\_file) | SSH public key file | `string` | `"~/.ssh/id_rsa.pub"` | no |
84+
| <a name="input_ssh_username"></a> [ssh\_username](#input\_ssh\_username) | SSH user, used only in output | `string` | `"root"` | no |
85+
| <a name="input_worker_os"></a> [worker\_os](#input\_worker\_os) | OS to run on worker machines | `string` | `"ubuntu"` | no |
86+
| <a name="input_workers_type"></a> [workers\_type](#input\_workers\_type) | GCE instance type | `string` | `"n1-standard-2"` | no |
87+
88+
## Outputs
89+
90+
| Name | Description |
91+
|------|-------------|
92+
| <a name="output_kubeone_api"></a> [kubeone\_api](#output\_kubeone\_api) | kube-apiserver LB endpoint |
93+
| <a name="output_kubeone_hosts"></a> [kubeone\_hosts](#output\_kubeone\_hosts) | Control plane endpoints to SSH to |
94+
| <a name="output_kubeone_workers"></a> [kubeone\_workers](#output\_kubeone\_workers) | Workers definitions, that will be transformed into MachineDeployment object |

‎examples/terraform/gce-ipv6/README.md.in

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# GCE Quickstart Terraform configs
2+
3+
The GCE Quickstart Terraform configs can be used to create the needed
4+
infrastructure for a Kubernetes HA cluster. Check out the following
5+
[Creating Infrastructure guide][docs-infrastructure] to learn more about how to
6+
use the configs and how to provision a Kubernetes cluster using KubeOne.
7+
8+
[docs-infrastructure]: https://docs.kubermatic.com/kubeone/v1.9/guides/using-terraform-configs/
9+
10+
## GCE Provider configuration
11+
12+
### Credentials
13+
14+
Per <https://www.terraform.io/docs/providers/google/provider_reference.html#configuration-reference>
15+
either of the following ENV variables should be accessible:
16+
17+
* `GOOGLE_CREDENTIALS`
18+
* `GOOGLE_CLOUD_KEYFILE_JSON`
19+
* `GCLOUD_KEYFILE_JSON`
20+

‎examples/terraform/gce-ipv6/main.tf

+208
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
/*
2+
Copyright 2019 The KubeOne Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
provider "google" {
18+
region = var.region
19+
project = var.project
20+
}
21+
22+
locals {
23+
zones_count = length(data.google_compute_zones.available.names)
24+
zone_first = data.google_compute_zones.available.names[0]
25+
kubeapi_endpoint = var.disable_kubeapi_loadbalancer ? google_compute_instance.control_plane.0.network_interface.0.network_ip : google_compute_address.lb_ip.0.address
26+
loadbalancer_count = var.disable_kubeapi_loadbalancer ? 0 : 1
27+
28+
cluster_autoscaler_min_replicas = var.cluster_autoscaler_min_replicas > 0 ? var.cluster_autoscaler_min_replicas : var.initial_machinedeployment_replicas
29+
cluster_autoscaler_max_replicas = var.cluster_autoscaler_max_replicas > 0 ? var.cluster_autoscaler_max_replicas : var.initial_machinedeployment_replicas
30+
}
31+
32+
data "google_compute_zones" "available" {
33+
}
34+
35+
data "google_compute_image" "control_plane_image" {
36+
family = var.control_plane_image_family
37+
project = var.control_plane_image_project
38+
}
39+
40+
resource "google_compute_network" "network" {
41+
name = var.cluster_name
42+
auto_create_subnetworks = false
43+
}
44+
45+
resource "google_compute_subnetwork" "subnet" {
46+
name = var.cluster_name
47+
network = google_compute_network.network.self_link
48+
ip_cidr_range = var.ip_cidr_range
49+
region = var.region
50+
stack_type = "IPV4_IPV6"
51+
ipv6_access_type = "EXTERNAL"
52+
}
53+
54+
resource "google_compute_firewall" "common" {
55+
name = "${var.cluster_name}-common"
56+
network = google_compute_network.network.self_link
57+
58+
allow {
59+
protocol = "tcp"
60+
ports = [var.ssh_port]
61+
}
62+
63+
source_ranges = [
64+
"0.0.0.0/0",
65+
]
66+
}
67+
68+
resource "google_compute_firewall" "control_plane" {
69+
name = "${var.cluster_name}-control-plane"
70+
network = google_compute_network.network.self_link
71+
72+
allow {
73+
protocol = "tcp"
74+
ports = ["6443"]
75+
}
76+
77+
source_ranges = [
78+
"0.0.0.0/0",
79+
]
80+
}
81+
82+
resource "google_compute_firewall" "internal" {
83+
name = "${var.cluster_name}-internal"
84+
network = google_compute_network.network.self_link
85+
86+
allow {
87+
protocol = "tcp"
88+
ports = ["0-65535"]
89+
}
90+
91+
allow {
92+
protocol = "udp"
93+
ports = ["0-65535"]
94+
}
95+
96+
allow {
97+
protocol = "icmp"
98+
}
99+
100+
source_ranges = [
101+
google_compute_subnetwork.subnet.ip_cidr_range,
102+
]
103+
}
104+
105+
resource "google_compute_firewall" "nodeports" {
106+
name = "${var.cluster_name}-nodeports"
107+
network = google_compute_network.network.self_link
108+
109+
allow {
110+
protocol = "tcp"
111+
ports = ["30000-32767"]
112+
}
113+
114+
source_ranges = [
115+
"0.0.0.0/0",
116+
]
117+
}
118+
119+
120+
resource "google_compute_address" "lb_ip" {
121+
count = local.loadbalancer_count
122+
123+
name = "${var.cluster_name}-lb-ip"
124+
}
125+
126+
resource "google_compute_http_health_check" "control_plane" {
127+
name = "${var.cluster_name}-control-plane-health"
128+
129+
port = 10256
130+
request_path = "/healthz"
131+
132+
timeout_sec = 3
133+
check_interval_sec = 5
134+
}
135+
136+
resource "google_compute_target_pool" "control_plane_pool" {
137+
name = "${var.cluster_name}-control-plane"
138+
139+
instances = slice(
140+
google_compute_instance.control_plane.*.self_link,
141+
0,
142+
var.control_plane_target_pool_members_count,
143+
)
144+
145+
health_checks = [
146+
google_compute_http_health_check.control_plane.self_link,
147+
]
148+
}
149+
150+
resource "google_compute_forwarding_rule" "control_plane" {
151+
count = local.loadbalancer_count
152+
153+
name = "${var.cluster_name}-apiserver"
154+
target = google_compute_target_pool.control_plane_pool.self_link
155+
port_range = "6443-6443"
156+
ip_address = google_compute_address.lb_ip.0.address
157+
}
158+
159+
resource "google_compute_instance" "control_plane" {
160+
count = var.control_plane_vm_count
161+
162+
name = "${var.cluster_name}-control-plane-${count.index + 1}"
163+
machine_type = var.control_plane_type
164+
zone = data.google_compute_zones.available.names[count.index % local.zones_count]
165+
166+
# Changing the machine_type, min_cpu_platform, or service_account on an
167+
# instance requires stopping it. To acknowledge this,
168+
# allow_stopping_for_update = true is required
169+
allow_stopping_for_update = true
170+
171+
boot_disk {
172+
initialize_params {
173+
size = var.control_plane_volume_size
174+
image = data.google_compute_image.control_plane_image.self_link
175+
}
176+
}
177+
178+
network_interface {
179+
subnetwork = google_compute_subnetwork.subnet.self_link
180+
181+
access_config {
182+
nat_ip = ""
183+
}
184+
185+
ipv6_access_config {
186+
network_tier = var.network_tier
187+
}
188+
189+
stack_type = "IPV4_IPV6"
190+
}
191+
192+
metadata = {
193+
sshKeys = "${var.ssh_username}:${file(var.ssh_public_key_file)}"
194+
}
195+
196+
# https://cloud.google.com/sdk/gcloud/reference/alpha/compute/instances/set-scopes#--scopes
197+
# listing of possible scopes
198+
service_account {
199+
scopes = [
200+
"compute-rw",
201+
"logging-write",
202+
"monitoring-write",
203+
"service-control",
204+
"service-management",
205+
"storage-ro",
206+
]
207+
}
208+
}

0 commit comments

Comments
 (0)