feat(comments): parser

[sh0rez]

Introduces a comment parser that extracts docsonnet data from specially formed comments, by leveraging as much of Jsonnet as possible:

• Comments are extracted from the snippet using Go code
• @<name> functions are defined in dsl.libsonnet as locals, so that using a bit of string replacement the comments can actually be evaluated. This keeps them compatible with Jsonnet syntax and avoids writing a custom parser :^)
• The evaluation result is pushed to the bottom of the snippet as a regular patch (Question: Should it be at the top instead, so it can be overwritten from Jsonnet?)

A new cmd/cparse currently exposes the functionality of parsing such Jsonnet:

  // @pkg("helm", "jsonnet-libs.org/helm")
  // @desc(|||
  //   Package `helm` converts Helm chart contents to regular Jsonnet
  //   objects to consume them with tools like [Grafana Tanka](https://tanka.dev)
  // |||)
  
  {
    // @fn("template")
    // @arg("name", string)
    // @arg("chart", string)
    // @arg("opts", object)
    // @desc("template renders a helm chart")
    template(name, chart, opts):: { /* ... */ }

TODO:

• Ignore non-docsonnet comments
• Be whitespace insensitive
• Write unit tests
• Integrate into docsonnet binary
• Write extensive docs how to use this
• Validate Function signature matches docsonnet

/cc @sbarzowski @ghostsquad

[ghostsquad]

Ooooo interesting

[ghostsquad]

The downside here is that you are putting the burden of making compatible comments on the user without the benefits of such things like autocomplete, or linting, since a comment is still a comment, yet it is evaluated like code. I'm not sure that's actually valuable

[sh0rez]

@ghostsquad Docsonnet follows the idea of structured documentation, which is similar to how tools like jsdoc work. By parsing a small DSL out of comments, it can present pretty powerful and information rich docs.

I experienced this to be more powerful than the ultra minimal approach done by godoc.

Things like autocomplete, syntax highlighting, etc., could definitely be added once we get to actually building editor tooling

[ghostsquad]

ya, ok, you convinced me. I'm wondering if we can reduce the syntax burden at all without painting ourselves into a corner. Example:

before

// @arg("opts", object)

after

// @arg object opts description?

arg 1: type
arg 2: name
arg 3: optional description, encourage default: foo here

this would unfortunately be a bit more difficult to code, but maybe it's easier to read and write?

[jeschkies]

This is really interesting indeed. I would have expected a minimal version like godoc. However, Jsonnet libs don't have a predefined structure so it makes sense to leave this up to the author and DSL they use.

Please excuse my ignorance but what is the output structure?

[malcolmholmes]

Please excuse my ignorance but what is the output structure?

What do you mean by "output structure"? This PR as I understand it converts comments to plain Jsonnet docsonnet, e.g.:

{
    "#myObj": d.obj("myObj holds my functions")
    myObj:: {
        "#myFunc": d.fn("myFunc greets you", [d.arg("who", d.T.string)])
        myFunc(who):: "hello %s!" % who
    }
}

From there, Docsonnet itself can generate HTML docs, e.g. as seen here.

[jeschkies]

Ah. I didn't know that bit was already there.