Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a maintenance process #116

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

funnelfiasco
Copy link
Contributor

This will undoubtedly require a lot of discussion before it's mergeable, but here's a first pass at addressing #86 and #96, which are related issues.

The gist of my initial draft is that instead of publishing the list of criteria directly to baseline.openssf.org, we publish it to a development version page. The index page has information about the project and links to versions. When we cut a new version, we copy the development.md to 'v1.2.md` or whatever we call that version.

I took the discussion in #86 as a starting point for the maintenance.md page, with one notable change: I described a process that's more continuous, with versions happening at least once a year (but more often if necessary), instead of keeping criteria sitting as open PRs for long periods.

We'll probably want to move some of the contents of README.md into the index.md file (specifically the description of the criteria) and include a link in README to the website instead, but that can be done later. We probably also want to add a "this is a work-in-progress. don't use it for evaluating your project" to the in-development output, and a version/date tag to the "published" versions.

@funnelfiasco funnelfiasco marked this pull request as draft December 18, 2024 19:19
@funnelfiasco
Copy link
Contributor Author

IMO we can proceed with this if the general shape is right, even if we need to continue iterating on the specifics of the maintenance process for a bit.

docs/maintenance.md Show resolved Hide resolved
docs/maintenance.md Show resolved Hide resolved
docs/maintenance.md Outdated Show resolved Hide resolved
Copy link
Contributor

@evankanderson evankanderson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I quite understand how the branching / releasing strategy will work for future versions -- will we be cutting a release branch for each baseline version? If so, should web-publish be checking out each release branch into a different sub-directory in the future?

(I'm not necessarily asking to do this now, I'm just trying to understand what's being communicated)

docs/maintenance.md Show resolved Hide resolved
docs/maintenance.md Outdated Show resolved Hide resolved
docs/maintenance.md Show resolved Hide resolved
docs/maintenance.md Show resolved Hide resolved
@funnelfiasco
Copy link
Contributor Author

I'm not sure I quite understand how the branching / releasing strategy will work for future versions -- will we be cutting a release branch for each baseline version? If so, should web-publish be checking out each release branch into a different sub-directory in the future?

(I'm not necessarily asking to do this now, I'm just trying to understand what's being communicated)

I don't plan on using branching. I can see the appeal, but I think it makes things like publishing the web content more difficult. So the idea is that when we decide we're releasing, we'd build the current state, check in the output as docs/versions/v<version number>. The build toolchain would treat it like a static page from there on out.

I didn't include the procedure here because I wanted to keep it to things that consumers care about. What do we think of a governance/SOP.md file that has the internal operations for things like this?

@SecurityCRob
Copy link
Contributor

thanks for putting this together @funnelfiasco . I'll put some thought to it and see if I come up with any new feedback beyond what the crew has given so far.

@SecurityCRob SecurityCRob added documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed labels Dec 18, 2024
@funnelfiasco funnelfiasco marked this pull request as ready for review January 21, 2025 22:03
@funnelfiasco funnelfiasco changed the title WIP: Add a maintenance process Add a maintenance process Jan 21, 2025
@funnelfiasco
Copy link
Contributor Author

Repeating from Slack: For the sake of expediency, let's focus on things that are broken or unclear. If something is missing, we can add it in a subsequent PR.

Copy link
Contributor

@evankanderson evankanderson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't consider my comments blocking. I suspect we'll end up needing to change these anyway as we go along, and this feels like a reasonable first pass.

docs/maintenance.md Outdated Show resolved Hide resolved
docs/maintenance.md Outdated Show resolved Hide resolved
Signed-off-by: Ben Cotton <[email protected]>
@funnelfiasco
Copy link
Contributor Author

Based on feedback, from Evan, I made a few minor wording changes:

diff --git a/docs/maintenance.md b/docs/maintenance.md
index ba604b8..6558c93 100644
--- a/docs/maintenance.md
+++ b/docs/maintenance.md
@@ -3,11 +3,11 @@
 * Normal text fixes to the criteria will be accepted via pull request and reviewed by the baseline project maintainers.
 Allowed changes are corrections to spelling/typos, grammar corrections, or enhancements to the supplementary text supporting the criteria, including: Objective, Implementation, Control Mappings, and Scorecard/Insights values.
 At least two project maintainers must review and approve these changes.
-* Substantive changes to Criteria, including changes to text that alters the originally stated meaning, new Criteria proposals, or removal of Criteria will be documented in GitHub PR(s) and reviewed regularly by the Baseline project maintainers.
+* Substantive changes to Criteria, including changes to text that alters the originally stated meaning, new Criteria proposals, or removal of Criteria will be documented in GitHub PR(s) and reviewed regularly by the Baseline project maintainers for inclusion in the next release.
 These changes may reflect changes to global cybersecurity regulations and frameworks or changes in norms around application/project security practices.
 Any such substantive changes must be approved by a majority of the project's maintainers.
 * As appropriate, but at least annually, the Baseline project maintainers will evaluate the set of criteria and, if necessary, publish a new version of the Baseline.
-Previous versions of the Baseline will remain avialable, but are not maintained.
+Previous versions of the Baseline will remain avialable, but are stable and not subject to change, except for minor changes to fix technical or typographic errors.
 * Any changes to the Baseline will be reflected within the Compliance Matrix, with new requirements flagged where the Baseline Criteria are appropriate.
 * Versions will follow a calendar-based identifcation system, using the `YYYY-MM-DD` format.
 * Downstream stakeholders will be notified via the project's mailing list on the changes and updates.

The only significant change is that I left us the option to fix typos/broken links, etc in released versions.

Also, I noticed a couple of typos that I have just fixed.

docs/index.md Outdated Show resolved Hide resolved
docs/maintenance.md Outdated Show resolved Hide resolved
funnelfiasco and others added 2 commits January 23, 2025 08:43
And also comment them out so that they aren't rendered until there are
actual things to put there.

Signed-off-by: Ben Cotton <[email protected]>
Comment on lines +21 to +22
* The numeric portion of identifiers are assigned sequentially per category.
They do not carry additional meaning.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SecurityCRob can you suggest a rephrase here to make it fit with your proposed three digit change? (ie, the levels DO carry additional meaning)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we fully set on the three digit change? The notes from last week say "try doing this with one category, and look at it." (although Crob has done all of the categories) and I, at least, am still not sold on the idea.

In the interests of getting this actually done, let's assume for now that the numbers aren't changing and we can update this if/when they do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants