|
| 1 | +# Development on puppet-agent module |
| 2 | + |
| 3 | +Welcome to the `puppet-agent` module! This is a document which will help |
| 4 | +guide you to getting started on development. The key sections here are: |
| 5 | + |
| 6 | +1. [Iterative code changes](#iterative) |
| 7 | +1. [Important files to understand repo](#files) |
| 8 | +1. [Other tools for development](#tools) |
| 9 | +1. [Testing](#testing) |
| 10 | + |
| 11 | +## Iterative code changes<a name="iterative"/> |
| 12 | + |
| 13 | +### Docker workflow |
| 14 | + |
| 15 | +Iterative development is the practice of reducing the amount of time |
| 16 | +between making a code change and seeing the result of that change. |
| 17 | +Below are a few methods to do this style of development. |
| 18 | + |
| 19 | +For iterative development using Docker, see: |
| 20 | +- docker/bin/upgrade.sh |
| 21 | +- docker/bin/versions.sh |
| 22 | +*Ubuntu* |
| 23 | +- docker/ubuntu/Dockerfile |
| 24 | +*CentOS* |
| 25 | +- docker/centos/Dockerfile |
| 26 | + |
| 27 | +*Note*: This will not start systemd services due to a limitation in |
| 28 | +Docker. It will upgrade packages properly, and should be useful for the |
| 29 | +majority of development on the module. |
| 30 | + |
| 31 | +### VMPooler workflow |
| 32 | + |
| 33 | +1. Forward your ssh key to do github operations. |
| 34 | +1. Ensure editor supports editing remote files. |
| 35 | +1. Checkout a vmpooler machine. |
| 36 | +1. Clone the module. |
| 37 | +1. Run `puppet apply` to test it as you work. |
| 38 | + |
| 39 | +### Development for Puppet Enterprise exclusive platforms |
| 40 | + |
| 41 | +#### Manual testing workflow |
| 42 | +1. Install an "old" PE on a master (typically RHEL). |
| 43 | +1. Declare the pe_repo classes for each agent you're testing upgrades for (e.g. like Windows, Fedora, etc.) using the PE Console. |
| 44 | +1. Install a "new" PE on the same master. |
| 45 | +1. Run puppet agent so that the pe_repo platforms have the new agent package. |
| 46 | +1. Copy local puppetlabs-puppet_agent module over to the /etc/puppetlabs/code/environment/production/modules directory. |
| 47 | +1. Install any module dependencies on the master via puppet module install (which you can get from the metadata.json file). |
| 48 | +1. Grab a VM to use as the agent, either in a local hypervisor or in VCloud. |
| 49 | +1. Test an upgrade scenario by installing the "old" agent by curling the old PE's install.bash script onto the agent VM and running puppet agent -t to link up with the master. Make sure to sign the agent’s CSR on the master. |
| 50 | +1. Declare the puppet_agent class in the site manifest (site.pp) declaring the master's agent as the puppet agent version for the agent node. Use the version returned by the master’s aio_agent_version fact. |
| 51 | +1. On the agent node, run `puppet agent -t`. |
| 52 | +1. To check that the upgrade was performed successfully, check the aio_agent_version fact on the agent to see if it matches what's reported on the master. |
| 53 | + |
| 54 | +## Important files to understand repo<a name="files"/> |
| 55 | + |
| 56 | +These are a few key files to look at to understand the basic flow of how the repository works: |
| 57 | + |
| 58 | +- manifests/init.pp -- This is the entrypoint for the module. This file |
| 59 | + describes the module's parameters, includes other classes(e.g. |
| 60 | + `prepare` and `install`), and adds some exception cases for a variety |
| 61 | + of platforms. |
| 62 | +- templates/do_install.sh.erb -- This calls the other scripts in the same directory. |
| 63 | + |
| 64 | +## Other tools for development<a name="tools"/> |
| 65 | + |
| 66 | +This ad-hoc job can be used to run CI against a branch |
| 67 | +[here](https://jenkins-master-prod-1.delivery.puppetlabs.net/view/puppet-agent/view/puppetlabs-puppet_agent%20module/view/ad-hoc/). |
| 68 | + |
| 69 | +This link/job may be updated with a new workflow in the near future. |
| 70 | + |
| 71 | +## Testing<a name="testing"/> |
| 72 | + |
| 73 | +Prior to committing any changes, ensure the tests pass locally. |
| 74 | + |
| 75 | +### Getting Started |
| 76 | + |
| 77 | +Our Puppet modules provide [`Gemfile`](./Gemfile)s, which can tell a Ruby package manager such as [bundler](http://bundler.io/) what Ruby packages, |
| 78 | +or Gems, are required to build, develop, and test this software. |
| 79 | + |
| 80 | +Please make sure you have [bundler installed](http://bundler.io/#getting-started) on your system, and then use it to |
| 81 | +install all dependencies needed for this project in the project root by running |
| 82 | + |
| 83 | +```shell |
| 84 | +% bundle install --path .bundle/gems |
| 85 | +Fetching gem metadata from https://rubygems.org/........ |
| 86 | +Fetching gem metadata from https://rubygems.org/.. |
| 87 | +Using rake (10.1.0) |
| 88 | +Using builder (3.2.2) |
| 89 | +-- 8><-- many more --><8 -- |
| 90 | +Using rspec-system-puppet (2.2.0) |
| 91 | +Using serverspec (0.6.3) |
| 92 | +Using rspec-system-serverspec (1.0.0) |
| 93 | +Using bundler (1.3.5) |
| 94 | +Your bundle is complete! |
| 95 | +Use `bundle show [gemname]` to see where a bundled gem is installed. |
| 96 | +``` |
| 97 | + |
| 98 | +NOTE: some systems may require you to run this command with sudo. |
| 99 | + |
| 100 | +If you already have those gems installed, make sure they are up-to-date: |
| 101 | + |
| 102 | +```shell |
| 103 | +% bundle update |
| 104 | +``` |
| 105 | + |
| 106 | +### Running Tests |
| 107 | + |
| 108 | +With all dependencies in place and up-to-date, run the tests: |
| 109 | + |
| 110 | +#### Unit Tests |
| 111 | + |
| 112 | +```shell |
| 113 | +% bundle exec rake spec |
| 114 | +``` |
| 115 | + |
| 116 | +This executes all the [rspec tests](http://rspec-puppet.com/) in the directories defined [here](https://github.com/puppetlabs/puppetlabs_spec_helper/blob/699d9fbca1d2489bff1736bb254bb7b7edb32c74/lib/puppetlabs_spec_helper/rake_tasks.rb#L17) and so on. |
| 117 | +rspec tests may have the same kind of dependencies as the module they are testing. Although the module defines these dependencies in its [metadata.json](./metadata.json), |
| 118 | +rspec tests define them in [.fixtures.yml](./fixtures.yml). |
| 119 | + |
| 120 | +#### Acceptance Tests |
| 121 | + |
| 122 | +Some Puppet modules also come with acceptance tests, which use [beaker][]. These tests spin up a virtual machine under |
| 123 | +[VirtualBox](https://www.virtualbox.org/), controlled with [Vagrant](http://www.vagrantup.com/), to simulate scripted test |
| 124 | +scenarios. In order to run these, you need both Virtualbox and Vagrant installed on your system. |
| 125 | + |
| 126 | +Run the tests by issuing the following command |
| 127 | + |
| 128 | +```shell |
| 129 | +% bundle exec rake spec_clean |
| 130 | +% bundle exec rspec spec/acceptance |
| 131 | +``` |
| 132 | + |
| 133 | +This will now download a pre-fabricated image configured in the [default node-set](./spec/acceptance/nodesets/default.yml), |
| 134 | +install Puppet, copy this module, and install its dependencies per [spec/spec_helper_acceptance.rb](./spec/spec_helper_acceptance.rb) |
| 135 | +and then run all the tests under [spec/acceptance](./spec/acceptance). |
| 136 | + |
| 137 | +### Writing Tests |
| 138 | + |
| 139 | +#### Unit Tests |
| 140 | + |
| 141 | +When writing unit tests for Puppet, [rspec-puppet][] is your best friend. It provides tons of helper methods for testing your manifests against a |
| 142 | +catalog (e.g. contain_file, contain_package, with_params, etc). It would be ridiculous to try and top rspec-puppet's [documentation][rspec-puppet_docs] |
| 143 | +but here's a tiny sample: |
| 144 | + |
| 145 | +Sample manifest: |
| 146 | + |
| 147 | +```puppet |
| 148 | +file { "a test file": |
| 149 | + ensure => present, |
| 150 | + path => "/etc/sample", |
| 151 | +} |
| 152 | +``` |
| 153 | + |
| 154 | +Sample test: |
| 155 | + |
| 156 | +```ruby |
| 157 | +it 'does a thing' do |
| 158 | + expect(subject).to contain_file("a test file").with({:path => "/etc/sample"}) |
| 159 | +end |
| 160 | +``` |
| 161 | + |
| 162 | +#### Acceptance Tests |
| 163 | + |
| 164 | +Writing acceptance tests for Puppet involves [beaker][] and its cousin [beaker-rspec][]. A common pattern for acceptance tests is to create a test manifest, apply it |
| 165 | +twice to check for idempotency or errors, then run expectations. |
| 166 | + |
| 167 | +More information about beaker and beaker-puppet is [here][https://github.com/puppetlabs/puppetlabs-puppet_agent/tree/master/acceptance]. |
| 168 | + |
| 169 | +```ruby |
| 170 | +it 'does an end-to-end thing' do |
| 171 | + pp = <<-EOF |
| 172 | + file { 'a test file': |
| 173 | + ensure => present, |
| 174 | + path => "/etc/sample", |
| 175 | + content => "test string", |
| 176 | + } |
| 177 | +
|
| 178 | + apply_manifest(pp, :catch_failures => true) |
| 179 | + apply_manifest(pp, :catch_changes => true) |
| 180 | +
|
| 181 | +end |
| 182 | +
|
| 183 | +describe file("/etc/sample") do |
| 184 | + it { is_expected.to contain "test string" } |
| 185 | +end |
| 186 | +
|
| 187 | +``` |
| 188 | +
|
| 189 | +## If you have commit access to the repository |
| 190 | +
|
| 191 | +Even if you have commit access to the repository, you still need to go through the process above, and have someone else review and merge |
| 192 | +in your changes. The rule is that **all changes must be reviewed by a project developer that did not write the code to ensure that |
| 193 | +all changes go through a code review process.** |
| 194 | +
|
| 195 | +The record of someone performing the merge is the record that they performed the code review. Again, this should be someone other than the author of the topic branch. |
| 196 | +
|
| 197 | +## Get Help |
| 198 | +
|
| 199 | +### On the web |
| 200 | +* [Puppet help messageboard](http://puppet.com/community/get-help) |
| 201 | +* [Writing tests](https://docs.puppet.com/guides/module_guides/bgtm.html#step-three-module-testing) |
| 202 | +* [General GitHub documentation](http://help.github.com/) |
| 203 | +* [GitHub pull request documentation](http://help.github.com/send-pull-requests/) |
| 204 | +
|
| 205 | +### On chat |
| 206 | +* Slack (slack.puppet.com) #forge-modules, #puppet-dev, #windows, #voxpupuli |
| 207 | +
|
| 208 | +
|
| 209 | +[rspec-puppet]: http://rspec-puppet.com/ |
| 210 | +[rspec-puppet_docs]: http://rspec-puppet.com/documentation/ |
| 211 | +[beaker]: https://github.com/puppetlabs/beaker |
| 212 | +[beaker-rspec]: https://github.com/puppetlabs/beaker-rspec |
0 commit comments