Skip to content

Commit 8453e7d

Browse files
kron4egkron4eg
committed
GCE dualstack support
This PR introduces a dedicated terraform config for GCE with IPv6 enabled Signed-off-by: Artiom Diomin <[email protected]>
1 parent 424dc85 commit 8453e7d

File tree

8 files changed

+660
-7
lines changed

8 files changed

+660
-7
lines changed

‎addons/cni-canal/canal-config.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,10 @@ data:
5353
{{ end }}
5454
},
5555
"policy": {
56-
"type": "k8s"
56+
"type": "k8s"
5757
},
5858
"kubernetes": {
59-
"kubeconfig": "__KUBECONFIG_FILEPATH__"
59+
"kubeconfig": "__KUBECONFIG_FILEPATH__"
6060
}
6161
},
6262
{

‎examples/terraform/gce-ipv6/README.md

+95
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# GCE Quickstart Terraform configs
2+
3+
The GCE Quickstart Terraform configs can be used to create the needed
4+
infrastructure for a Kubernetes HA cluster. Check out the following
5+
[Creating Infrastructure guide][docs-infrastructure] to learn more about how to
6+
use the configs and how to provision a Kubernetes cluster using KubeOne.
7+
8+
[docs-infrastructure]: https://docs.kubermatic.com/kubeone/v1.9/guides/using-terraform-configs/
9+
10+
## GCE Provider configuration
11+
12+
### Credentials
13+
14+
Per <https://www.terraform.io/docs/providers/google/provider_reference.html#configuration-reference>
15+
either of the following ENV variables should be accessible:
16+
17+
* `GOOGLE_CREDENTIALS`
18+
* `GOOGLE_CLOUD_KEYFILE_JSON`
19+
* `GCLOUD_KEYFILE_JSON`
20+
21+
## Requirements
22+
23+
| Name | Version |
24+
|------|---------|
25+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0.0 |
26+
| <a name="requirement_google"></a> [google](#requirement\_google) | ~> 6 |
27+
28+
## Providers
29+
30+
| Name | Version |
31+
|------|---------|
32+
| <a name="provider_google"></a> [google](#provider\_google) | ~> 6 |
33+
34+
## Modules
35+
36+
No modules.
37+
38+
## Resources
39+
40+
| Name | Type |
41+
|------|------|
42+
| [google_compute_address.lb_ip](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_address) | resource |
43+
| [google_compute_firewall.common](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
44+
| [google_compute_firewall.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
45+
| [google_compute_firewall.internal](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
46+
| [google_compute_firewall.nodeports](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |
47+
| [google_compute_forwarding_rule.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_forwarding_rule) | resource |
48+
| [google_compute_http_health_check.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_http_health_check) | resource |
49+
| [google_compute_instance.control_plane](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance) | resource |
50+
| [google_compute_network.network](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_network) | resource |
51+
| [google_compute_subnetwork.subnet](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_subnetwork) | resource |
52+
| [google_compute_target_pool.control_plane_pool](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_target_pool) | resource |
53+
| [google_compute_image.control_plane_image](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_image) | data source |
54+
| [google_compute_zones.available](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/compute_zones) | data source |
55+
56+
## Inputs
57+
58+
| Name | Description | Type | Default | Required |
59+
|------|-------------|------|---------|:--------:|
60+
| <a name="input_apiserver_alternative_names"></a> [apiserver\_alternative\_names](#input\_apiserver\_alternative\_names) | subject alternative names for the API Server signing cert. | `list(string)` | `[]` | no |
61+
| <a name="input_bastion_host_key"></a> [bastion\_host\_key](#input\_bastion\_host\_key) | Bastion SSH host public key | `string` | `null` | no |
62+
| <a name="input_cluster_autoscaler_max_replicas"></a> [cluster\_autoscaler\_max\_replicas](#input\_cluster\_autoscaler\_max\_replicas) | maximum number of replicas per MachineDeployment (requires cluster-autoscaler) | `number` | `0` | no |
63+
| <a name="input_cluster_autoscaler_min_replicas"></a> [cluster\_autoscaler\_min\_replicas](#input\_cluster\_autoscaler\_min\_replicas) | minimum number of replicas per MachineDeployment (requires cluster-autoscaler) | `number` | `0` | no |
64+
| <a name="input_cluster_name"></a> [cluster\_name](#input\_cluster\_name) | Name of the cluster | `string` | n/a | yes |
65+
| <a name="input_control_plane_image_family"></a> [control\_plane\_image\_family](#input\_control\_plane\_image\_family) | Image family to use for provisioning instances | `string` | `"ubuntu-2404-lts-amd64"` | no |
66+
| <a name="input_control_plane_image_project"></a> [control\_plane\_image\_project](#input\_control\_plane\_image\_project) | Project of the image to use for provisioning instances | `string` | `"ubuntu-os-cloud"` | no |
67+
| <a name="input_control_plane_target_pool_members_count"></a> [control\_plane\_target\_pool\_members\_count](#input\_control\_plane\_target\_pool\_members\_count) | n/a | `number` | `3` | no |
68+
| <a name="input_control_plane_type"></a> [control\_plane\_type](#input\_control\_plane\_type) | GCE instance type | `string` | `"n1-standard-2"` | no |
69+
| <a name="input_control_plane_vm_count"></a> [control\_plane\_vm\_count](#input\_control\_plane\_vm\_count) | number of control plane instances | `number` | `3` | no |
70+
| <a name="input_control_plane_volume_size"></a> [control\_plane\_volume\_size](#input\_control\_plane\_volume\_size) | Size of the boot volume, in GB | `number` | `100` | no |
71+
| <a name="input_disable_kubeapi_loadbalancer"></a> [disable\_kubeapi\_loadbalancer](#input\_disable\_kubeapi\_loadbalancer) | E2E tests specific variable to disable usage of any loadbalancer in front of kubeapi-server | `bool` | `false` | no |
72+
| <a name="input_enable_ula_internal_ipv6"></a> [enable\_ula\_internal\_ipv6](#input\_enable\_ula\_internal\_ipv6) | Enable ULA internal ipv6 on this network. Enabling this feature will assign a /48 from google defined ULA prefix fd20::/20 | `bool` | `false` | no |
73+
| <a name="input_initial_machinedeployment_operating_system_profile"></a> [initial\_machinedeployment\_operating\_system\_profile](#input\_initial\_machinedeployment\_operating\_system\_profile) | Name of operating system profile for MachineDeployments, only applicable if operating-system-manager addon is enabled.<br>If not specified, the default value will be added by machine-controller addon. | `string` | `""` | no |
74+
| <a name="input_initial_machinedeployment_replicas"></a> [initial\_machinedeployment\_replicas](#input\_initial\_machinedeployment\_replicas) | Number of replicas per MachineDeployment | `number` | `2` | no |
75+
| <a name="input_ip_cidr_range"></a> [ip\_cidr\_range](#input\_ip\_cidr\_range) | The range of internal addresses that are owned by this subnetwork. Ranges must<br>be unique and non-overlapping within a network. Only IPv4 is supported.value | `string` | `"10.255.0.0/16"` | no |
76+
| <a name="input_ipv6_access_type"></a> [ipv6\_access\_type](#input\_ipv6\_access\_type) | The access type of IPv6 address this subnet holds. It's immutable and can only be specified during<br>creation or the first time the subnet is updated into IPV4\_IPV6 dual stack. If the ipv6\_type is<br>EXTERNAL then this subnet cannot enable direct path. Possible values: ["EXTERNAL", "INTERNAL"] | `string` | `"EXTERNAL"` | no |
77+
| <a name="input_network_tier"></a> [network\_tier](#input\_network\_tier) | The service-level to be provided for IPv6 traffic when the subnet has an external subnet.<br>Only PREMIUM or STANDARD tier is valid for IPv6. | `string` | `"PREMIUM"` | no |
78+
| <a name="input_project"></a> [project](#input\_project) | Project to be used for all resources | `string` | n/a | yes |
79+
| <a name="input_region"></a> [region](#input\_region) | GCP region to speak to | `string` | `"europe-west3"` | no |
80+
| <a name="input_ssh_agent_socket"></a> [ssh\_agent\_socket](#input\_ssh\_agent\_socket) | SSH Agent socket, default to grab from $SSH\_AUTH\_SOCK | `string` | `"env:SSH_AUTH_SOCK"` | no |
81+
| <a name="input_ssh_hosts_keys"></a> [ssh\_hosts\_keys](#input\_ssh\_hosts\_keys) | A list of SSH hosts public keys to verify | `list(string)` | `null` | no |
82+
| <a name="input_ssh_port"></a> [ssh\_port](#input\_ssh\_port) | SSH port to be used to provision instances | `number` | `22` | no |
83+
| <a name="input_ssh_private_key_file"></a> [ssh\_private\_key\_file](#input\_ssh\_private\_key\_file) | SSH private key file used to access instances | `string` | `""` | no |
84+
| <a name="input_ssh_public_key_file"></a> [ssh\_public\_key\_file](#input\_ssh\_public\_key\_file) | SSH public key file | `string` | `"~/.ssh/id_rsa.pub"` | no |
85+
| <a name="input_ssh_username"></a> [ssh\_username](#input\_ssh\_username) | SSH user, used only in output | `string` | `"root"` | no |
86+
| <a name="input_worker_os"></a> [worker\_os](#input\_worker\_os) | OS to run on worker machines | `string` | `"ubuntu"` | no |
87+
| <a name="input_workers_type"></a> [workers\_type](#input\_workers\_type) | GCE instance type | `string` | `"n1-standard-2"` | no |
88+
89+
## Outputs
90+
91+
| Name | Description |
92+
|------|-------------|
93+
| <a name="output_kubeone_api"></a> [kubeone\_api](#output\_kubeone\_api) | kube-apiserver LB endpoint |
94+
| <a name="output_kubeone_hosts"></a> [kubeone\_hosts](#output\_kubeone\_hosts) | Control plane endpoints to SSH to |
95+
| <a name="output_kubeone_workers"></a> [kubeone\_workers](#output\_kubeone\_workers) | Workers definitions, that will be transformed into MachineDeployment object |

‎examples/terraform/gce-ipv6/README.md.in

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# GCE Quickstart Terraform configs
2+
3+
The GCE Quickstart Terraform configs can be used to create the needed
4+
infrastructure for a Kubernetes HA cluster. Check out the following
5+
[Creating Infrastructure guide][docs-infrastructure] to learn more about how to
6+
use the configs and how to provision a Kubernetes cluster using KubeOne.
7+
8+
[docs-infrastructure]: https://docs.kubermatic.com/kubeone/v1.9/guides/using-terraform-configs/
9+
10+
## GCE Provider configuration
11+
12+
### Credentials
13+
14+
Per <https://www.terraform.io/docs/providers/google/provider_reference.html#configuration-reference>
15+
either of the following ENV variables should be accessible:
16+
17+
* `GOOGLE_CREDENTIALS`
18+
* `GOOGLE_CLOUD_KEYFILE_JSON`
19+
* `GCLOUD_KEYFILE_JSON`
20+

‎examples/terraform/gce-ipv6/main.tf

+209
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,209 @@
1+
/*
2+
Copyright 2019 The KubeOne Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
provider "google" {
18+
region = var.region
19+
project = var.project
20+
}
21+
22+
locals {
23+
zones_count = length(data.google_compute_zones.available.names)
24+
zone_first = data.google_compute_zones.available.names[0]
25+
kubeapi_endpoint = var.disable_kubeapi_loadbalancer ? google_compute_instance.control_plane.0.network_interface.0.network_ip : google_compute_address.lb_ip.0.address
26+
loadbalancer_count = var.disable_kubeapi_loadbalancer ? 0 : 1
27+
28+
cluster_autoscaler_min_replicas = var.cluster_autoscaler_min_replicas > 0 ? var.cluster_autoscaler_min_replicas : var.initial_machinedeployment_replicas
29+
cluster_autoscaler_max_replicas = var.cluster_autoscaler_max_replicas > 0 ? var.cluster_autoscaler_max_replicas : var.initial_machinedeployment_replicas
30+
}
31+
32+
data "google_compute_zones" "available" {
33+
}
34+
35+
data "google_compute_image" "control_plane_image" {
36+
family = var.control_plane_image_family
37+
project = var.control_plane_image_project
38+
}
39+
40+
resource "google_compute_network" "network" {
41+
name = var.cluster_name
42+
auto_create_subnetworks = false
43+
enable_ula_internal_ipv6 = var.enable_ula_internal_ipv6
44+
}
45+
46+
resource "google_compute_subnetwork" "subnet" {
47+
name = var.cluster_name
48+
network = google_compute_network.network.self_link
49+
ip_cidr_range = var.ip_cidr_range
50+
region = var.region
51+
stack_type = "IPV4_IPV6"
52+
ipv6_access_type = var.ipv6_access_type
53+
}
54+
55+
resource "google_compute_firewall" "common" {
56+
name = "${var.cluster_name}-common"
57+
network = google_compute_network.network.self_link
58+
59+
allow {
60+
protocol = "tcp"
61+
ports = [var.ssh_port]
62+
}
63+
64+
source_ranges = [
65+
"0.0.0.0/0",
66+
]
67+
}
68+
69+
resource "google_compute_firewall" "control_plane" {
70+
name = "${var.cluster_name}-control-plane"
71+
network = google_compute_network.network.self_link
72+
73+
allow {
74+
protocol = "tcp"
75+
ports = ["6443"]
76+
}
77+
78+
source_ranges = [
79+
"0.0.0.0/0",
80+
]
81+
}
82+
83+
resource "google_compute_firewall" "internal" {
84+
name = "${var.cluster_name}-internal"
85+
network = google_compute_network.network.self_link
86+
87+
allow {
88+
protocol = "tcp"
89+
ports = ["0-65535"]
90+
}
91+
92+
allow {
93+
protocol = "udp"
94+
ports = ["0-65535"]
95+
}
96+
97+
allow {
98+
protocol = "icmp"
99+
}
100+
101+
source_ranges = [
102+
google_compute_subnetwork.subnet.ip_cidr_range,
103+
]
104+
}
105+
106+
resource "google_compute_firewall" "nodeports" {
107+
name = "${var.cluster_name}-nodeports"
108+
network = google_compute_network.network.self_link
109+
110+
allow {
111+
protocol = "tcp"
112+
ports = ["30000-32767"]
113+
}
114+
115+
source_ranges = [
116+
"0.0.0.0/0",
117+
]
118+
}
119+
120+
121+
resource "google_compute_address" "lb_ip" {
122+
count = local.loadbalancer_count
123+
124+
name = "${var.cluster_name}-lb-ip"
125+
}
126+
127+
resource "google_compute_http_health_check" "control_plane" {
128+
name = "${var.cluster_name}-control-plane-health"
129+
130+
port = 10256
131+
request_path = "/healthz"
132+
133+
timeout_sec = 3
134+
check_interval_sec = 5
135+
}
136+
137+
resource "google_compute_target_pool" "control_plane_pool" {
138+
name = "${var.cluster_name}-control-plane"
139+
140+
instances = slice(
141+
google_compute_instance.control_plane.*.self_link,
142+
0,
143+
var.control_plane_target_pool_members_count,
144+
)
145+
146+
health_checks = [
147+
google_compute_http_health_check.control_plane.self_link,
148+
]
149+
}
150+
151+
resource "google_compute_forwarding_rule" "control_plane" {
152+
count = local.loadbalancer_count
153+
154+
name = "${var.cluster_name}-apiserver"
155+
target = google_compute_target_pool.control_plane_pool.self_link
156+
port_range = "6443-6443"
157+
ip_address = google_compute_address.lb_ip.0.address
158+
}
159+
160+
resource "google_compute_instance" "control_plane" {
161+
count = var.control_plane_vm_count
162+
163+
name = "${var.cluster_name}-control-plane-${count.index + 1}"
164+
machine_type = var.control_plane_type
165+
zone = data.google_compute_zones.available.names[count.index % local.zones_count]
166+
167+
# Changing the machine_type, min_cpu_platform, or service_account on an
168+
# instance requires stopping it. To acknowledge this,
169+
# allow_stopping_for_update = true is required
170+
allow_stopping_for_update = true
171+
172+
boot_disk {
173+
initialize_params {
174+
size = var.control_plane_volume_size
175+
image = data.google_compute_image.control_plane_image.self_link
176+
}
177+
}
178+
179+
network_interface {
180+
subnetwork = google_compute_subnetwork.subnet.self_link
181+
182+
access_config {
183+
nat_ip = ""
184+
}
185+
186+
ipv6_access_config {
187+
network_tier = var.network_tier
188+
}
189+
190+
stack_type = "IPV4_IPV6"
191+
}
192+
193+
metadata = {
194+
sshKeys = "${var.ssh_username}:${file(var.ssh_public_key_file)}"
195+
}
196+
197+
# https://cloud.google.com/sdk/gcloud/reference/alpha/compute/instances/set-scopes#--scopes
198+
# listing of possible scopes
199+
service_account {
200+
scopes = [
201+
"compute-rw",
202+
"logging-write",
203+
"monitoring-write",
204+
"service-control",
205+
"service-management",
206+
"storage-ro",
207+
]
208+
}
209+
}

0 commit comments

Comments
 (0)