Skip to content
This repository has been archived by the owner on Sep 18, 2020. It is now read-only.

Commit

Permalink
*: add support for ignition when using virtualbox
Browse files Browse the repository at this point in the history
Ignition support for virtualbox is present in Container Linux
versions >= 1451.0.0
  • Loading branch information
Alexander Pavel committed Jul 11, 2017
1 parent 19af1c3 commit 3dcc151
Show file tree
Hide file tree
Showing 6 changed files with 141 additions and 22 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,7 @@
log/
user-data
config.rb
config.ign
config.ign.merged
*.vmdk
*.img
28 changes: 24 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,10 +76,28 @@ config.vm.synced_folder ".", "/home/core/share", id: "core", :nfs => true, :mou

After a 'vagrant reload' you will be prompted for your local machine password.

#### Provisioning with user-data
#### Provisioning with Ignition (VirtualBox Provider (default))

The Vagrantfile will provision your CoreOS VM(s) with [coreos-cloudinit][coreos-cloudinit] if a `user-data` file is found in the project directory.
coreos-cloudinit simplifies the provisioning process through the use of a script or cloud-config document.
When using the VirtualBox provider for Vagrant (the default), Ignition is used to provision the machine. This uses a special plugin that is
automatically installed when using the default Vagrantfile. The config options for the plugin are all prefixed with `config.ignition` and can
be found in this Vagrantfile or in the README of the [plugin](https://github.com/coreos/vagrant-ignition)

To get started, run `curl https://discovery.etcd.io/new\?size\=X`, where `X` is the number of servers in your cluster (if a size is not provided,
the default of 3 will be used). More configuration may be added if necessary. Then, use config transpiler to write the Ignition config
to config.ign by running `ct --platform=vagrant-virtualbox < cl.conf > config.ign`. To see all available configuration options, check out
the [Container Linux Configuration Specification][clspec] as well as the [Container Linux Config Transpiler Getting Started Documentation][ignition].
There is also a basic Ignition file provided based on the Container Linux config that is included. To use that instead (not recommended),
copy `config.ign.sample` to `config.ign` and make any necessary modifications. Check out the [Ignition Getting Started documentation][ignition]
to learn about the available features.

[ignition]: https://github.com/coreos/docs/blob/master/os/provisioning.md
[clspec]: https://github.com/coreos/container-linux-config-transpiler/blob/master/doc/configuration.md

#### Provisioning with user-data (VMWare provider)

When using the VMWare provider for Vagrant, the Vagrantfile will provision your CoreOS VM(s)
with [coreos-cloudinit][coreos-cloudinit] if a `user-data` file is found in the project directory. coreos-cloudinit simplifies the
provisioning process through the use of a script or cloud-config document.

To get started, copy `user-data.sample` to `user-data` and make any necessary modifications.
Check out the [coreos-cloudinit documentation][coreos-cloudinit] to learn about the available features.
Expand All @@ -94,7 +112,9 @@ See `config.rb.sample` for more information.
## Cluster Setup

Launching a CoreOS cluster on Vagrant is as simple as configuring `$num_instances` in a `config.rb` file to 3 (or more!) and running `vagrant up`.
Make sure you provide a fresh discovery URL in your `user-data` if you wish to bootstrap etcd in your cluster.
If using the VirtualBox provider (default), copy the make sure to create a `config.ign` as described above so that the machines can be configured with
etcd and flanneld correctly. Also, make sure to provide a fresh discovery URL in your `config.ign` file to bootstrap etcd in your cluster.
If you are using the VMWare provider, make sure you provide a fresh discovery URL in your `user-data` if you wish to bootstrap etcd in your cluster.

## New Box Versions

Expand Down
41 changes: 33 additions & 8 deletions Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,26 @@ require 'fileutils'

Vagrant.require_version ">= 1.6.0"

# Make sure the vagrant-ignition plugin is installed
required_plugins = %w(vagrant-ignition)

plugins_to_install = required_plugins.select { |plugin| not Vagrant.has_plugin? plugin }
if not plugins_to_install.empty?
puts "Installing plugins: #{plugins_to_install.join(' ')}"
if system "vagrant plugin install #{plugins_to_install.join(' ')}"
exec "vagrant #{ARGV.join(' ')}"
else
abort "Installation of one or more plugins has failed. Aborting."
end
end

CLOUD_CONFIG_PATH = File.join(File.dirname(__FILE__), "user-data")
IGNITION_CONFIG_PATH = File.join(File.dirname(__FILE__), "config.ign")
CONFIG = File.join(File.dirname(__FILE__), "config.rb")

# Defaults for config options defined in CONFIG
$num_instances = 1
$instance_name_prefix = "core"
$update_channel = "alpha"
$image_version = "current"
$enable_serial_logging = false
$share_home = false
$vm_gui = false
Expand Down Expand Up @@ -51,15 +63,12 @@ Vagrant.configure("2") do |config|
# forward ssh agent to easily ssh into the different machines
config.ssh.forward_agent = true

config.vm.box = "coreos-%s" % $update_channel
if $image_version != "current"
config.vm.box_version = $image_version
end
config.vm.box_url = "https://storage.googleapis.com/%s.release.core-os.net/amd64-usr/%s/coreos_production_vagrant.json" % [$update_channel, $image_version]
config.vm.box = "coreos-alpha"
config.vm.box_url = "https://alpha.release.core-os.net/amd64-usr/current/coreos_production_vagrant_virtualbox.json"

["vmware_fusion", "vmware_workstation"].each do |vmware|
config.vm.provider vmware do |v, override|
override.vm.box_url = "https://storage.googleapis.com/%s.release.core-os.net/amd64-usr/%s/coreos_production_vagrant_vmware_fusion.json" % [$update_channel, $image_version]
override.vm.box_url = "https://alpha.release.core-os.net/amd64-usr/current/coreos_production_vagrant_vmware_fusion.json"
end
end

Expand All @@ -68,6 +77,8 @@ Vagrant.configure("2") do |config|
# in CoreOS, so tell Vagrant that so it can be smarter.
v.check_guest_additions = false
v.functional_vboxsf = false
# enable ignition (this is always done on virtualbox as this is how the ssh key is added to the system)
config.ignition.enabled = true
end

# plugin conflict
Expand Down Expand Up @@ -122,10 +133,13 @@ Vagrant.configure("2") do |config|
vb.memory = vm_memory
vb.cpus = vm_cpus
vb.customize ["modifyvm", :id, "--cpuexecutioncap", "#{$vb_cpuexecutioncap}"]
config.ignition.config_obj = vb
end

ip = "172.17.8.#{i+100}"
config.vm.network :private_network, ip: ip
# This tells Ignition what the IP for eth1 (the host-only adapter) should be
config.ignition.ip = ip

# Uncomment below to enable NFS for sharing the host machine into the coreos-vagrant VM.
#config.vm.synced_folder ".", "/home/core/share", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']
Expand All @@ -137,11 +151,22 @@ Vagrant.configure("2") do |config|
config.vm.synced_folder ENV['HOME'], ENV['HOME'], id: "home", :nfs => true, :mount_options => ['nolock,vers=3,udp']
end

# This shouldn't be used for the virtualbox provider (it doesn't have any effect if it is though)
if File.exist?(CLOUD_CONFIG_PATH)
config.vm.provision :file, :source => "#{CLOUD_CONFIG_PATH}", :destination => "/tmp/vagrantfile-user-data"
config.vm.provision :shell, :inline => "mv /tmp/vagrantfile-user-data /var/lib/coreos-vagrant/", :privileged => true
end

config.vm.provider :virtualbox do |vb|
config.ignition.hostname = vm_name
config.ignition.config_vmdk = File.join(File.dirname(__FILE__), "config" + i.to_s + ".vmdk")
config.ignition.config_img = "config" + i.to_s + ".img"
# when the ignition config doesn't exist, the plugin automatically generates a very basic Ignition with the ssh key
# and previously specified options (ip and hostname). Otherwise, it appends those to the provided config.ign below
if File.exist?(IGNITION_CONFIG_PATH)
config.ignition.path = 'config.ign'
end
end
end
end
end
33 changes: 33 additions & 0 deletions cl.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
etcd:
name: "{HOSTNAME}"
listen_peer_urls: "http://{PRIVATE_IPV4}:2380"
listen_client_urls: "http://0.0.0.0:2379"
initial_advertise_peer_urls: "http://{PRIVATE_IPV4}:2380"
advertise_client_urls: "http://{PRIVATE_IPV4}:2379"
# replace "<token>" with a valid etcd discovery token
discovery: "https://discovery.etcd.io/<token>"

systemd:
units:
- name: docker-tcp.socket
enable: true
contents: |
[Unit]
Description=Docker Socket for the API

[Socket]
ListenStream=2375
Service=docker.service
BindIPv6Only=both

[Install]
WantedBy=sockets.target
- name: flanneld.service
dropins:
- name: 50-network-config.conf
contents: |
[Service]
ExecStartPre=/usr/bin/etcdctl set /flannel/network/config '{ "Network": "10.1.0.0/16" }'

flannel:
etcd_prefix: "/flannel/network"
47 changes: 47 additions & 0 deletions config.ign.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"ignition": {
"version": "2.0.0",
"config": {}
},
"storage": {},
"systemd": {
"units": [
{
"name": "etcd-member.service",
"enable": true,
"dropins": [
{
"name": "20-clct-etcd-member.conf",
"contents": "[Unit]\nRequires=coreos-metadata.service\nAfter=coreos-metadata.service\n\n[Service]\nEnvironmentFile=/run/metadata/coreos\nExecStart=\nExecStart=/usr/lib/coreos/etcd-wrapper $ETCD_OPTS \\\n --name=\"${COREOS_VAGRANT_VIRTUALBOX_HOSTNAME}\" \\\n --listen-peer-urls=\"http://${COREOS_VAGRANT_VIRTUALBOX_PRIVATE_IPV4}:2380\" \\\n --listen-client-urls=\"http://0.0.0.0:2379\" \\\n --initial-advertise-peer-urls=\"http://${COREOS_VAGRANT_VIRTUALBOX_PRIVATE_IPV4}:2380\" \\\n --advertise-client-urls=\"http://${COREOS_VAGRANT_VIRTUALBOX_PRIVATE_IPV4}:2379\" \\\n --discovery=\"https://discovery.etcd.io/<token>\""
}
]
},
{
"name": "flanneld.service",
"enable": true,
"dropins": [
{
"name": "20-clct-flannel.conf",
"contents": "[Service]\nExecStart=\nExecStart=/usr/lib/coreos/flannel-wrapper $FLANNEL_OPTS \\\n --etcd-prefix=\"/flannel/network\""
}
]
},
{
"name": "docker-tcp.socket",
"enable": true,
"contents": "[Unit]\nDescription=Docker Socket for the API\n\n[Socket]\nListenStream=2375\nService=docker.service\nBindIPv6Only=both\n\n[Install]\nWantedBy=sockets.target\n"
},
{
"name": "flanneld.service",
"dropins": [
{
"name": "50-network-config.conf",
"contents": "[Service]\nExecStartPre=/usr/bin/etcdctl set /flannel/network/config '{ \"Network\": \"10.1.0.0/16\" }'\n"
}
]
}
]
},
"networkd": {},
"passwd": {}
}
10 changes: 0 additions & 10 deletions config.rb.sample
Original file line number Diff line number Diff line change
Expand Up @@ -45,16 +45,6 @@ end
# "core-01" through to "core-${num_instances}".
#$instance_name_prefix="core"

# Change the version of CoreOS to be installed
# To deploy a specific version, simply set $image_version accordingly.
# For example, to deploy version 709.0.0, set $image_version="709.0.0".
# The default value is "current", which points to the current version
# of the selected channel
#$image_version = "current"

# Official CoreOS channel from which updates should be downloaded
#$update_channel='alpha'

# Log the serial consoles of CoreOS VMs to log/
# Enable by setting value to true, disable with false
# WARNING: Serial logging is known to result in extremely high CPU usage with
Expand Down

3 comments on commit 3dcc151

@jamesooo
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this commit is also locking the repo to the alpha core os image instead of using what the user provides. am i wrong or does documentation just need to be updated to reflect this change?

@AlexNPavel
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct. This was due to the support for VirtualBox only being in the latest alpha (now it was promoted to beta). We may also add support for Ignition for the VMWare provider, which would likely keep us locked to alpha until that release makes it to stable as well. Once both providers have Ignition support in the stable channel of Container Linux, we can add the channel configuration back again.

@jamesooo
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I knew there had to be a reason for it. Thanks for the quick response

Please sign in to comment.