Feature Request: Gomplate as a Template Manager

[osterman]

@hairyhenderson I understand this may be a long shot feature request! Anyways, just wanted to get it on your radar, in case it sounds cool.

what

• support a way to create templatized projects, e.g. a full-fledged template manager for boilerplate code
• provide a common way to reference project templates e.g. using github repos
• support a project config file with properties (variables) and a github project source for the template.

why

• Sometimes more "abstractions" is just more convoluted when simple template generation does the job. Template generation is very well understood. It allows projects to diverge, but also provides a way to systematically update by regenerating templates
• The boilr project appears abandoned
• Gomplate is insanely powerful for templating

our use-cases

Don't get me wrong, gomplate can do this today with some scripting. We do it already to some degree at cloudposse. Just I would love if gomplate became a convention for how to structure templatized projects, rather than everyone creating their own convention. E.g. today, we use a Makefile with a bunch of custom scripting.

1. We generate our README.md from a README.yaml
2. Generate the LICENSE file from the projects configuration
3. Automatically create .github/workflows (or travis/circle/codefresh pipelines) for a project with template parameters
4. Create the scaffolding for "terraform" modules

prior art

• https://github.com/tmrts/boilr
  
• example boilr templates https://github.com/tmrts/boilr/wiki/Templates

[hairyhenderson]

Hey @osterman! Thanks for this, it's really interesting 🤔

First off, I'm going to assume that full boilr functionality isn't what you're looking for necessarily - is the interactive prompting stuff important? If so I'm thinking that probably belongs in a separate higher-level gomplate-based app. Guiding users interactively is useful to be sure, but IMO it doesn't make sense with the gomplate flow right now...

Barring the interactivity, this sounds to me like a smarter kind of --input-dir that can understand a conventional layout that includes a set of default values that don't need to be explicitly provided with a --context or --datasource argument. An interesting feature of boilr is that it understands templated filenames too - something that gomplate currently doesn't - that could be part of this, though TBH I can't see why it couldn't just as a matter of course. Though it'd be pretty weird to have a files named {{ .foo | regexp.Replace "(foo)bar" "$1" }}.

I'm not sure --input-dir is right, but maybe a new --input-project argument is all that's needed, something like:

# grab the project from the filesystem
$ gomplate --input-project ./sample-project
...
# grab the project from GitHub
$ gomplate --input-project github.com/hairyhenderson/sample-project
...
# or some other git repo
$ gomplate --input-project [email protected]:hairyhenderson/sample-project.git
...

These would all use the defaults provided in the project, probably using that file as the . context, I guess, and then overriding seems fairly natural:

$ gomplate --context .=./my_overrides.yaml --input-project github.com/hairyhenderson/sample-project

Am I missing anything?

[osterman]

@hairyhenderson yes, I think what you describe is pretty accurate.

I'm going to assume that full boilr functionality isn't what you're looking for necessarily - is the interactive prompting stuff important?

True, but I think it does look fancy and demos well =)

Barring the interactivity, this sounds to me like a smarter kind of --input-dir

Agree.

that includes a set of default values that don't need to be explicitly provided

Yes, some sane defaults, like values.yaml in helm.

You suggest: (I agree)

gomplate --context .=./my_overrides.yaml --input-project github.com/hairyhenderson/sample-project

Now, the only modifications I would make are....

1. Add a --config=GOMPLATE_CONFIG parameter that defaults to something. GOMPLATE_CONFIG is an environment variable and if not defined, defaults to .gomplate.yaml or something.
2. We can specify all arguments in this config.
3. Support subpaths and tags for version pinning and monorepos like in helmfile or terraform. e.g.

• --input-project github.com/hairyhenderson/sample-project@foo/bar?ref=1.2.3 (helmfile syntax)
• --input-project github.com/hairyhenderson/sample-project//foo/bar?ref=1.2.3 (terraform syntax)

For example, .gomplate.yaml:

context: ./my_overrides.yaml 
input-project: github.com/hairyhenderson/sample-project//foo/bar?ref=1.2.3
output-dir: .
overwrite: true

Now, in a project, we can just run gomplate and it will render the project, overwriting any files that t conflict. Running git diff (assuming committed code) will show us what changed. We can always revert or cherry-pick changes.

An interesting feature of boilr is that it understands templated filenames too - something that gomplate currently doesn't

I think this is nice, but could maybe come later?

[hairyhenderson]

(eep, I thought I responded days ago - somehow this didn't get posted!)

@osterman ok, yeah, that all sounds awesome. I've been thinking about a gomplate config file recently, and it'd fit nicely into this.

I'm going to try to break this down into bite-sized chunks, which should mostly be able to be implemented atomically. I'll create separate issues for these as I go to make them easier to work on:

• Support a gomplate config file to set commandline (and other!) params
  • defaults to .gomplate.yaml, overridden by GOMPLATE_CONFIG, and finally by --config=
• Support for boilerplate projects with --input-project (or maybe just --project?)
  • layout of templates/ folder plus defaults.yaml file
• Support reading projects from git repositories, including subpaths and git refs
  • maybe this should extend to --file and --input-dir too?
  • maybe this should be more generic support for reading from URLs (using the same schemes as for datasources)?

[rossedman]

@hairyhenderson 👋 also very interested in this. was going to recommend the above (config file, reading projects from git, etc). I was wondering if this work had begun or what was the status of it. Thanks.

[hairyhenderson]

@rossedman hi! I've taken a bit of a break from gomplate for the past month or so - not for any particular reason, just lots of "life" things taking precedence 😉.

All that to say, no I haven't begun working on this, though I was just thinking about it the other day. I won't promise any kind of timeline, but I will promise that I'll get started on this sometime soon. I ran into a few situations over the past couple weeks where at least the git support would've been useful.

[hairyhenderson]

Just a bit of an update - support for git datasources, which is tangentially related to this feature, is pretty close to dropping here: #661 (comments welcome, especially on the docs!). It was a bit more of a pain to implement than I expected 😂

[lorengordon]

This would tie in very nicely to a gomplate github action also... Have a git project managing the templates, and a github action where gomplate uses that project as the template source, templates the files in the current repo, and opens a pull request with any changes.

[github-actions]

This issue is stale because it has been open for 60 days with no activity. Remove stale label or comment or this will be automatically closed in a few days.

[osterman]

Would still love to see this! =)

[github-actions]

This issue is stale because it has been open for 60 days with no
activity. If it is no longer relevant or necessary, please close it.
Given no action, it will be closed in 14 days.

If it's still relevant, one of the following will remove the stale
marking:

• A maintainer can add this issue to a milestone to indicate that
  it's been accepted and will be worked on
• A maintainer can remove the stale label
• Anyone can post an update or other comment

[hairyhenderson]

I've assigned this to the future milestone so this shouldn't go stale again. This might be fulfilled by v4, but I'm not certain.