Support ecosystem severity under severity[]

[litios]

Currently, the schema allows CVSS severities under severity[].
Nevertheless, you can find references to other severities under other
areas:

• On the summary (AlmaLinux, RockyLinux)
• Per-package under ecosystem_specific (Android, Ubuntu)
• Under database_specific (Bitnami, GitHub issues)
• Under references (RedHat)

It's common practice for security teams to provide a severity related to
the security issue within their ecosystem. The severities previously
mentioned are examples of them.

This presents a problem within the schema: publishers have the need to
provide this ecosystem severity, but there is no defined way to do so.
This results in having several references to severity within the
document and no standard way to retrieve this ecosystem severity, since
each publisher is taking a different approach.

Given that severity[] already exists, it would make sense to have an
ECOSYSTEM type severity. This is the same type used for providing
version ranges, so this wouldn't be unexpected. A namespace optional
field could be used to provide a reference to the meaning of the score
as per the ecosystem, similar to how it's done in CVE records.

An example using Red Hat namespace:

"severity": [
    {
      "type": "ECOSYSTEM",
      "score": "Low",
      "namespace": "https://access.redhat.com/security/updates/classification"
    }
]

This proposal was developed with feedback from Red Hat and Ubuntu, that
already support this request.

[oliverchang]

Thanks for the proposal! And sorry for the slow response -- just coming back after the holiday period.

Our initial intent was indeed to rely on database_specific or ecosystem_specific to encode qualitative severities, since the meaning of them likely depend on the data source. Since we now have a severity field, it might make sense to make this more easily encoded in a more consistent way across different sources.

Regarding the actual semantics of this proposal, we have two severity fields: a top level one and a package-specific one under affected[]. type: ECOSYSTEM would only make sense under the latter. I wonder if we should have this as type: DATABASE instead that applies to both?

Adding in some other distro PoCs for comment:

@Roo4L (AlmaLinux)
@luhring (Chainguard/Wolfi)
@mstg (Rocky Linux)
@jasinner (Red Hat)
@msmeissn (SUSE)
@dodys (Ubuntu)

And also @darakian @andrewpollock (other schema maintainers).

[litios]

@oliverchang thanks for the feedback!

I want to clarify this part:

type: ECOSYSTEM would only make sense under the latter. I wonder if we should have this as type: DATABASE instead that applies to both?

ECOSYSTEM is being used for packages with a meaning that is similar as to what we are aiming for here: the information is specific to the ecosystem referenced.

The severity definitions are defined by the ecosystem. For example, the definition of a Red Hat severity of Important won't change across databases, as it is defined by Red Hat.

Just to be clear, I'm not against your proposal but given that ECOSYSTEM is already covered by the spec and seems to fit the same goal, I want to clarify why it's not a valid option.

[oliverchang]

Thanks for the clarifying your rationale!

My main concern is that there isn't a concept of a top level ecosystem in OSV records -- there may be many ecosystems encoded within a single record under multiple affected[] entries. Having an top level severity type named ECOSYSTEM severity makes things potentially a little inconsistent/confusing here.

Perhaps QUALITATIVE might be a better word here that better communicates this severity type?

{
  "id": "ID-1234",
  "severity": [
    {
      "type": "QUALITITATIVE",
      "score": "Low",
      "namespace": "https://access.redhat.com/security/updates/classification"
    }
  ]
}

[litios]

The idea of this new severity type is not to provide a single top-level one but to use it just as another type. Probably, most cases would only have one instance (the one from the publisher) but multiple different ones could live in the same report, so it would be no different than the affected[] entries.

Following with your example, let's say Oracle Linux provides its severity but also wants to provides Red Hat one, because in this case there is a difference and that wants to be stated. Both of them would coexists under severity[] as different entries.

{
  "id": "ID-1234",
  "severity": [
    {
      "type": "ECOSYSTEM",
      "score": "Low",
      "namespace": "https://access.redhat.com/security/updates/classification"
    },
    {
      "type": "ECOSYSTEM",
      "score": "Moderate",
      "namespace": "https://oracle-linux/security-ratings"
    },
    {
      "type": "CVSS_V4",
      "score": "CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:L/VI:L/VA:L/SC:N/SI:N/SA:N"
    }
  ],
  "affected": [
    {
      "package": {
      ...
}

About QUALITITATIVE, that sounds good to me too!
I just want to clarify about the ECOSYSTEM one first, just to be sure I'm explaining the idea correctly.

[Roo4L]

Hi! I am no longer maintaining OSV entries in AlmaLinux, please contact @andrewlukoshko about this.

[oliverchang]

The idea of this new severity type is not to provide a single top-level one but to use it just as another type. Probably, most cases would only have one instance (the one from the publisher) but multiple different ones could live in the same report, so it would be no different than the affected[] entries.

Yep, I understand that :) My suggestion essentially boils down to renaming ECOSYSTEM to QUALITATIVE because ECOSYSTEM potentially clashes in meaning with existing ECOSYSTEM for describing package ecosystems.

[litios]

Thanks for the clarifying! If it's mainly for the purpose of not mixing the meanings I'm happy with QUALITATIVE.

Looking at next steps, would PRs from our end be preferred or would your team handle this internally?

[oliverchang]

Thanks for the clarifying! If it's mainly for the purpose of not mixing the meanings I'm happy with QUALITATIVE.

Looking at next steps, would PRs from our end be preferred or would your team handle this internally?

A PR would be great! What we'd also like to see before merging the PR though, is some feedback from other Linux distro database maintainers also.

[darakian]

Also coming back from some holiday time (and a bit sick 🤒). If I understand this correctly the basic idea is to allow for more severity systems with a link out to the rules for that new system? Is that an accurate gist? If so how many of these system do we expect to add and maybe they could be added as types akin to how cvss 3 and 4 are types?

[oliverchang]

Also coming back from some holiday time (and a bit sick 🤒). If I understand this correctly the basic idea is to allow for more severity systems with a link out to the rules for that new system? Is that an accurate gist?

Yep! Specifically qualitative ratings like "High", "Critical", because they're highly subjective measures that are very specific to a particular source/database.

If so how many of these system do we expect to add and maybe they could be added as types akin to how cvss 3 and 4 are types?

I'd imagine there would be a lot and too many to list in the schema, since there's not really a standard definition for "High", "Critical", "Medium", "Low". Every vulnerability database could have its own definition.

[jasinner]

Is it correct that we'd be able to use the new QUALITATIVE severity in both top level and for each package? I think that would be ideal to be able to describe Red Hat's assessed impact for the top level erratum impact (which is the highest of the package severities) and the individual packages themselves, which is often different.

[darakian]

I'd imagine there would be a lot and too many to list in the schema

Do we expect to have tens to hundreds of severity rating systems? Maybe I guess, but I feel like throwing these all behind a url makes the job of the reader much harder over time as urls come and go.

[litios]

Do we expect to have tens to hundreds of severity rating systems?

I don't think we will get to hundreds but tens sounds realistic. In most cases, a record will have the QUALITATIVE severity from the publisher itself and, in some remote cases, maybe another related to this. Nothing stops a publisher from providing others but that's the most realistic scenario IMO

as urls come and go.

This is part of the motivation of having it as a field itself and not as key-value type that users need to check in the documentation. It all comes down to maintaining those URLs. If they are on the records, the publisher can handle it. Otherwise, a request to change the documentation to point to the new URL (or to add a new one) would be needed for each case.

[litios]

Is it correct that we'd be able to use the new QUALITATIVE severity in both top level and for each package?

Package severity mentions that If any package level severity fields are set, the top level severity must not be set. In order to provide both on the same record, the schema would need to change, regardless of this new QUALITATIVE addition.

[darakian]

It seems more ideal to me to add new types (and docs) to this repo for each of the new severity ranking systems so that URL maintenance isn't even a concern. I agree there is friction there, but in my opinion it's worth it. I know @andrewpollock is out at the moment, but I'd love to hear his thoughts on this as well.

[oliverchang]

(Note that @andrewpollock is OOO for a few weeks).

I do agree with @darakian that if the number of qualitative schemes are in the tens, then it's worthwhile maintaining an explicit list in the schema.

This is also more consistent with how we enumerate specs for other types in the OSV schema (e.g. package ecosystem types). WDYT @litios ?

[litios]

Having each severity as an actual severity type sounds good to me too. The proposal was to avoid overloading OSV team with handling all of these severities but I agree it would make the schema more consistent and the reading/parsing/displaying simpler.

Just to be sure I understood correctly, about:

add new types (and docs) to this repo for each of the new severity ranking systems so that URL maintenance isn't even a concern

are we talking about adding the description of each of the severity ratings for each of the new severity types to the documentation? If this is the approach, I understand that each publisher wanting to add a new severity type would need to provide this information so it's added to the table and ensuring it's up to date when changes are made. Is this correct?

[dodys]

sorry to jump in late on this, the year started with quite a lot.

Regarding the proposal, Ubuntu/Canonical has been in talks with David/Oracle since the start and from reading all the discussion here I think we are going in the correct direction. I do like the QUALITATIVE naming.

About the severity field being either on the top or in per package, the guideline is that if we have different severities per package that we don't set one in the top, is that correct? And that doesn't change with the QUALITATIVE right?

Regarding keeping a list of URLs explaining the different definitions of different vendors here, does it mean we also won't include it into the OSV record? Or should we have it in both places?

[darakian]

What I'm envisioning at least would not require a new QUALITATIVE field, but could look something like

{
  "id": "ID-1234",
  "severity": [
    {
      "type": "RedHatsCoolSeveritySystem_V1",
      "score": "Low",

    },
    {
      "type": "UbuntusHotSeveritySystem_V1",
      "score": "Moderate",
    },
    {
      "type": "CVSS_V4",
      "score": "CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:L/VI:L/VA:L/SC:N/SI:N/SA:N"
    }
  ],
  "affected": [
    {
      "package": {
      ...
}

Where RedHatsCoolSeveritySystem_V1 and UbuntusHotSeveritySystem_V1 (or whatever real names get made) are defined in the osv schema similar to the different CVSS types

osv-schema/docs/schema.md

Lines 600 to 610 in 2cac7e8

	### severity[].type field
	The `severity[].type` property must be one of the types defined below, which
	describes the quantitative method used to calculate the associated `score`.
	| Severity Type | Score Description |
	| --------- | ----------- |
	| `CVSS_V2` | A CVSS vector string representing the unique characteristics and severity of the vulnerability using a version of the [Common Vulnerability Scoring System notation](https://www.first.org/cvss/v2/guide#Metric-Groups) that is == 2.0 (e.g.`"AV:L/AC:M/Au:N/C:N/I:P/A:C"`).|
	| `CVSS_V3` | A CVSS vector string representing the unique characteristics and severity of the vulnerability using a version of the [Common Vulnerability Scoring System notation](https://www.first.org/cvss/v3.0/specification-document#Vector-String) that is >= 3.0 and < 4.0 (e.g.`"CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:C/C:H/I:N/A:N"`).|
	| `CVSS_V4` | A CVSS vector string representing the unique characteristics and severity of the vulnerability using a version on the [Common Vulnerability Scoring System notation](https://www.first.org/cvss/v4.0/specification-document#Vector-String) that is >= 4.0 and < 5.0 (e.g. `"CVSS:4.0/AV:N/AC:L/AT:N/PR:H/UI:N/VC:L/VI:L/VA:N/SC:N/SI:N/SA:N"`). |
	| Your quantitative severity type here. | [Send us a PR](https://github.com/ossf/osv-schema/compare). |

[oliverchang]

if we have different severities per package that we don't set one in the top, is that correct? And that doesn't change with the QUALITATIVE right?

That's correct.

Does Ubuntu also have a desire to express both a top-level and per-package severities? @jasinner also expressed a desire to have both a top level and per-package severities in #323 (comment) for Red Hat.

Re naming, I'd suggest just defaulting to the existing naming for package ecosystems for the types for consistency, unless there's a reason to be different. e.g.

{
  "id": "ID-1234",
  "severity": [
    {
      "type": "Red Hat",
      "score": "Low",
    },
    {
      "type": "Ubuntu",
      "score": "Low",
    },
    {
      "type": "CVSS_V4",
      "score": "CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:L/VI:L/VA:L/SC:N/SI:N/SA:N"
    }
  ],
}

@darakian Would GitHub would also need its own?

Regarding keeping a list of URLs explaining the different definitions of different vendors here, does it mean we also won't include it into the OSV record? Or should we have it in both places?

The intention is to just keep the schema as the single source of truth and define the explanations in the schema. Is there are a reason to prefer otherwise?

[dodys]

if we have different severities per package that we don't set one in the top, is that correct? And that doesn't change with the QUALITATIVE right?

That's correct.

Does Ubuntu also have a desire to express both a top-level and per-package severities? @jasinner also expressed a desire to have both a top level and per-package severities in #323 (comment) for Red Hat.

We don't normally have many cases of Ubuntu priorities that are different per package in a same CVE, but we do support it, and we would only mention the priority for the package that is different from the top priority. If in OSV instead of doing that, we want to be more explicit, that's fine by us, not an issue.

Re naming, I'd suggest just defaulting to the existing naming for package ecosystems for the types for consistency, unless there's a reason to be different. e.g.

{
  "id": "ID-1234",
  "severity": [
    {
      "type": "Red Hat",
      "score": "Low",
    },
    {
      "type": "Ubuntu",
      "score": "Low",
    },
    {
      "type": "CVSS_V4",
      "score": "CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:L/VI:L/VA:L/SC:N/SI:N/SA:N"
    }
  ],
}

That's even better!

@darakian Would GitHub would also need its own?

Regarding keeping a list of URLs explaining the different definitions of different vendors here, does it mean we also won't include it into the OSV record? Or should we have it in both places?

The intention is to just keep the schema as the single source of truth and define the explanations in the schema. Is there are a reason to prefer otherwise?

Not from me, the way you described above looks good to me :)
The only thing though is that, at least for Ubuntu, we do give an explanation when we "deviate" from CVSS for example, and we have that already OSV in the ecosystem as priority_reason. Would we keep doing as is or would we have that field under severities?
@litios thoughts on this ^

[darakian]

@darakian Would GitHub would also need its own?

No, I think we're good with CVSS for the foreseeable future. Perhaps someday in the future we might look at an independent severity system, but I think we would want it to used beyond us as well so let me get back to you in a decade or two on that :)

[litios]

From my side, I'm happy with the format pointed out by @oliverchang, keeping the names aligned with the ecosystem names makes it clear. It will also make it simpler from the documentation side.

As per what @dodys mentioned:

The only thing though is that, at least for Ubuntu, we do give an explanation when we "deviate" from CVSS for example, and we have that already OSV in the ecosystem as priority_reason. Would we keep doing as is or would we have that field under severities?

I know sometimes Red Hat provides an statement too (example) and others might follow and provide reason for the severity. It would probably need to be renamed to severity_reason or similar though and set as an optional field.

[oliverchang]

Thanks all for the discussions! Any other comments around what's proposed in #323 (comment)?

@andrewlukoshko (AlmaLinux)
@luhring (Chainguard/Wolfi)
@mstg (Rocky Linux)
@jasinner (Red Hat)
@msmeissn (SUSE)
@dodys (Ubuntu)

[luhring]

That format looks good to me!

We don't have our own severity system today—we defer to upstream data—but it's great to know that we could have one if we wanted.

Small question: I saw mention of a _V1 suffix up above. is the idea that if an ecosystem wants to be able to iterate on how its severities are expressed in OSV, it's the ecosystem's responsibility to version (or somehow uniquely identify) each incarnation of how severities work for them with a unique "type" value here, and then document them all in the OSV schema? Or is the intent of Oliver's comment that, no, we're not doing that suffix thing, each "type" should simply be the name of the ecosystem, and we'll accept that there wouldn't be a "safe" way to change the ecosystem's severity system in OSV? (I'm good either way, just making sure I understand!)

[darakian]

Small question: I saw mention of a _V1 suffix up above. is the idea that if an ecosystem wants to be able to iterate on how its severities are expressed in OSV, it's the ecosystem's responsibility to version (or somehow uniquely identify) each incarnation of how severities work for them with a unique "type" value here, and then document them all in the OSV schema?

Assuming that question is directed at me; ya basically. Severity is a contextual concept and as the contexts of advisories change over time it may make sense to change how a severity is derived. As a reader of advisories its helpful to know what the rules/guidelines lead to a given score and hence versioning and writing them down was what I had in my head.

That is somewhat heavy weight in terms of process though. I am not opposed to there being an ecosystem default ruleset (which could change over time) which would be maintained by the ecosystem owner(s) and I don't think the two ideas are incompatible either.

[litios]

Since it looks like the issue is being stalled for a bit now -- @oliverchang what would be the next steps here?

We talked about providing a PR from my side in the past but I'm not sure there is much I can provide right now. As we decided to go for each publisher having their severity type under the schema, I understand each publisher would need to provide their own information/update to the documentation in order for the type to be recorded? And probably update the documentation to provide a brief introduction about these types and what they refer to.

Let me know if there is anything I can do from my end!

[tiegz]

re "QUALITATIVE": aren't CVSS vector strings qualitative themselves? They're a list of qualitative variables that can get transformed into a quantitative score.

[edit: apologies in case "QUALITATIVE" has already been ruled out]

[oliverchang]

Yes, as you mention @tiegz, we have agreement that "QUALITATIVE" is no longer the way to go.

@litios: Yes, it appears everybody is aligned on ecosystem-specific severities:

{
  "id": "ID-1234",
  "severity": [
    {
      "type": "Red Hat",
      "score": "Low",
    },
    {
      "type": "Ubuntu",
      "score": "Low",
    },
    {
      "type": "CVSS_V4",
      "score": "CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:L/VI:L/VA:L/SC:N/SI:N/SA:N"
    }
  ],
}

The next steps here would be for relevant distros to add definitions to the schema and to start publishing this. @dodys @jasinner given your input in this issue, would you be open to creating PRs to define these for your relevant distros?

[dodys]

@oliverchang done, let me know if there's anything else I missed