Merge 1.7.0 into live branch. #342
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Use a structured error for package version findings - tidy up the passing around of configurations - add support for only checking specific ecosystems Signed-off-by: Andrew Pollock <[email protected]> --------- Signed-off-by: Andrew Pollock <[email protected]>
Some old flaws have no 'scores' field at all. In that case create the CSAF object with the cvss_v3_attributes set to an empty string. --------- Signed-off-by: Jason Shepherd <[email protected]>
Also add Wolfi to the documented ecosystems (unfortunately this will need another release) Signed-off-by: Andrew Pollock <[email protected]>
Will link from OSSF blog. Signed-off-by: Jason Shepherd <[email protected]>
@andrewpollock As discussed, removing version and other qualifiers from package PURL in Red Hat data. --------- Signed-off-by: Jason Shepherd <[email protected]>
Including the arch suffix in a fixed version can cause false positives if the wrong arch range is used to scan a system. For example `0:7.3.0-427.28.1.el9_4.aarch64` < `0:7.3.0-427.28.1.el9_4.ppc64le` So `0:7.3.0-427.28.1.el9_4.aarch64` would match the `0..0:7.3.0-427.28.1.el9_4.ppc64le` range and be considered vulnerable Signed-off-by: Jason Shepherd <[email protected]>
These are all very minor but still typos none the less Signed-off-by: Gareth Jones <[email protected]>
This setups up a workflow for running the `osv-linter` tests for pull requests - it looks like there are also tests for the Python-based tools, which I'll look into as a follow up. This'll make it easier to later add CI for #292 Signed-off-by: Gareth Jones <[email protected]>
`affected:` should be defined in the data source and should not be an empty array. --------- Signed-off-by: Holly Gong <[email protected]>
This is the current latest version, with none of the breaking changes being impactful though [v44.0.0's is noteworthy for forks](https://github.com/tj-actions/changed-files/releases/tag/v44.0.0) Signed-off-by: Gareth Jones <[email protected]>
Among other things, this should help ensure that editors use the same indenting - note that currently a lot of the Python files in particular are formatted with 4 spaces instead of 2, which will be taken care of at a later date Signed-off-by: Gareth Jones <[email protected]>
For records like RHSA-2024:8116, `.affected[]` was ending up with the packages multiple times Signed-off-by: Andrew Pollock <[email protected]>
Sometimes RPMs are packed in a module. It's a little harder to detect the actual modules themselves on a running system, but we can still check their RPMs. This change allows skipping modules, but keeps their RPM children. Signed-off-by: Jason Shepherd <[email protected]>
The table is pretty important but currently you can't actually link directly to it, so this adds a header for that Signed-off-by: Gareth Jones <[email protected]>
This introduces new dedicated types for the name of an ecosystem (which is an enum) and an ecosystem suffix (which is a string starting with `:`), along with renaming the existing `ecosystem` type to `ecosystemWithSuffix` (+ adding it). While these types are not actually used in the schema due to JSONSchema currently does not have a way of specifying that a string is made up of two types, having them defined enables a lot of automation as the current `pattern` based type cannot be used to get a finite list of all possible ecosystems defined by the spec. As an example, here's a [TypeScript playground](https://www.typescriptlang.org/play/?#code/C4TwDgpgBAogxgewM4icCBbKBeKByOAJwEN0kA6ASwTygB98BxG+-AWWIDcIA7PAWABQQ0JFiIUaTAHVKwABYAFQhABmlAB45xyVOiwMABgBIA3vF1SMAXwBcZtIUo8A5tYfAnr64YDcQoQATCDgAG2IVKFUAVx44YGoeKJ4ACg1bHUl9WQVlNU0ASgzOBEpA-2FBVVSCEjIqBFsARgAmPAKhapTa0ggKanbOmqJe-sbByud0QlViOGhFCKQIQIsszChTISgoEMt9DLW9TAqdpGjVdXSoR2cXCusgA) showcasing how you could use the types that would be automatically generated from this new schema structure using [`json-schema-to-typescript`](https://www.npmjs.com/package/json-schema-to-typescript) (with a manually defined `EcosystemSuffix` type, though we can later introduce that for TypeScript specifically by adding a `tsType` property). Notably, landing this will unblock me adding automation to ensure that all lists of the ecosystem name are in sync (notably, the table in the docs, the enum + pattern here, and soon the constants being added in #292) Signed-off-by: Gareth Jones <[email protected]>
This migrates some of the "schema" based types that currently live in `osv-scanner` over to here since they're related to the underlying OSV schema rather than the scanner specifically. Notably to help avoid dependency cycles we now have a dedicated `constants` package whose sole responsibility is to house types and constants for representing enums defined in the schema with no logic whatsoever; this will ensure that this package is always a leaf in the dependency tree, preventing circular dependencies in libraries (which notably we have now in the current `osv-scanner/models` package). This also introduces an `ecosystem` package which houses our concept of an parsed ecosystem as implicitly defined in the spec - that is, a struct made up of an ecosystem name and an optional suffix. Note the underlying `Ecosystem` type actually lives in `constants` which might seem weird at first but 1. prevents cycles as mentioned above, and 2. avoids the weird `ecosystem.Ecosystem` situation. Currently this is being introduced as a port from `osv-scanner` - the fact that there are missing constants will be addressed in a follow-up pull request, along with other changes such as updating the `validation/schema.json` and (hopefully) creating some automation to help keep everything in sync. --------- Signed-off-by: Gareth Jones <[email protected]>
PURLs should not include the `@version` component when used in OSV. `affected[].ranges[]` should be used for this purpose. Signed-off-by: Oliver Chang <[email protected]>
This reformates the JSON schema file to reduce the diff on #303 since currently there's a section that is incorrectly indented Signed-off-by: Gareth Jones <[email protected]>
This introduces a new script to make it easier to ensure all "lists of ecosystems" within this codebase remain up to date, including: - the table in `docs/schema.md` - the Go constants being introduced in #292 - the JSON schema in `validation/schema.json` (both the pattern and the enum being introduced in #296) To make it a bit easier, I've introduced a top-level `ecosystems.json` which is a map of defined ecosystems and a markdown description, sorted alphabetically (which the script also ensures) - I felt this was easier than trying to extract the list from markdown or another source, though it does mean double quotes need to be manually escaped. I went with JSON as it can be read without requiring an external dependency, though if we use Python 3.11 we could switch to `toml` instead as that ships with `tomllib` Example of the workflow output:   --------- Signed-off-by: Gareth Jones <[email protected]>
This commit restructures the test data so that CSAF and OSV data is not co-mingled in the same directory. It is an attept to address ``` $ go install github.com/ossf/osv-schema/tools/osv-linter@latest go: downloading github.com/ossf/osv-schema v1.6.7 go: github.com/ossf/osv-schema/tools/osv-linter@latest: create zip: tools/redhat/testdata/rhsa-2024_4546.json: case-insensitive file name collision: "tools/redhat/testdata/RHSA-2024_4546.json" and "tools/redhat/testdata/rhsa-2024_4546.json" ``` --------- Signed-off-by: Andrew Pollock <[email protected]>
This commit adds a README.md to provide interested parties with some further details and guidance about the OSV linter Signed-off-by: Andrew Pollock <[email protected]>
This is defined in the schema but we don't have a constant for it Signed-off-by: Gareth Jones <[email protected]>
Red Hat changed the document.category on 31st Oct. Therefore any records since then have not been processed. See https://access.redhat.com/articles/5554431 Signed-off-by: Jason Shepherd <[email protected]>
Adds Red Hat and SUSE/openSUSE and sorts consistent with case insensitive sort of OSV.dev's `ecosystems.txt` Signed-off-by: Andrew Pollock <[email protected]>
This corrects a bug where the ecosystem allowlist was not working correctly when not set, and so ecosystem checks were not being run at all. Signed-off-by: Andrew Pollock <[email protected]>
Fixes #249. --------- Signed-off-by: Oliver Chang <[email protected]> Signed-off-by: Oliver Chang <[email protected]> Co-authored-by: Dan Luhring <[email protected]>
…lusivity (#327) `affected[].severity` mentions: 'If any package level severity fields are set, the top level severity must not be set.' This change updates the JSON schema under validation to reflect this. Changes have been tested with both tools mentioned under validation/README.md --------- Signed-off-by: David Fernandez Gonzalez <[email protected]> Co-authored-by: Oliver Chang <[email protected]>
I first discovered this repo from a link over at Hackernews, but it wasn't clear what OSV was. After clicking around a bit I found this statement on the [osv.dev](https://osv.dev/) homepage: <img src="https://github.com/user-attachments/assets/4ad51a5f-d3f6-46e0-a3ef-6372133164e5" /> This PR simply adds the highlighted short description to the top of this repo's README. This should help new readers get a quick and basic understanding of what OSV is without having to leave this repo. If the maintainers of this repo feel that this PR isn't necessary, just go ahead and close it -- you won't hurt my feelings 😄 --------- Signed-off-by: Johnny <[email protected]> Signed-off-by: Oliver Chang <[email protected]> Co-authored-by: Oliver Chang <[email protected]>
Adding `Kubernetes` ecosystem according to the discussion [here](kubernetes-sigs/cve-feed-osv#9). This PR does not specify how to handle Kubernetes distributions by cloud vendors like EKS and GKE. We can talk about it here if needed, or merge it once and discuss it separately. --------- Signed-off-by: knqyf263 <[email protected]> Signed-off-by: Teppei Fukuda <[email protected]> Co-authored-by: Oliver Chang <[email protected]>
This commit adds the Mageia prefix `MGASA` to acceptable prefixes so records pass schema validation. Signed-off-by: Andrew Pollock <[email protected]>
This commit adds the `upstream` field to the schema definition. It was omitted from #312 Signed-off-by: Andrew Pollock <[email protected]>
Moving the struct definitions of the OSV record from [osv-scanner](https://github.com/google/osv-scanner/blob/b870d62b9987f27ab0212a9fd0eeef8086554699/pkg/models/vulnerability.go) to this repository, since it makes more sense to live in the `osvschema` module. I've also added the new `upstream` field to the struct. It's probably worth having a discussion on how to make sure these structs remain in-sync with any future changes to the schema. --------- Signed-off-by: Michael Kedar <[email protected]> Co-authored-by: Andrew Pollock <[email protected]>
These seem to be unnecessary. Signed-off-by: Michael Kedar <[email protected]>
This PR enhances the linter by adding package existence checks for a variety of ecosystems: - crates.io - npm - NuGet - RubyGems - Packagist - Pub - Hackage - Maven To accommodate these additions, the `ecosystems.go` file has been refactored, with code related to package existence checks moved to `package_check.go` and version checks moved to `version_check.go`. A current limitation is that malicious package record validation may fail, as these packages are often removed from package registries and are no longer queryable via API. --------- Signed-off-by: Holly Gong <[email protected]>
Fix a confusing typo from "affected filed" to "affected field". Signed-off-by: Viviana Sutedjo <[email protected]>
### Add V8 prefix to OSV-schema. As per internal discussions, we are adding the "V8" prefix for policy-based synthetic vulnerability feeds to OSV schema. The Chromium team has a [policy](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/security/updates.md) that states that direct embedders of Chromium packages such as V8 are responsible for using an up-to-date version of V8 packages. The definition of that is that the V8 package is at most one week old. Since these types of packages do not have their own identifiers and therefore no advisory could be created, enforcement of such policies was not possible. We now created a synthetic advisory generator that takes such time-based policies and converts them into regularily updated advisories that OSV can read: https://github.com/google/chromium-policy-vulnfeed/blob/main/advisories/V8-advisory.json Refer to @oliverchang for context. Signed-off-by: Viviana Sutedjo <[email protected]>
Relates to issue #323 --------- Signed-off-by: Eduardo Barretto <[email protected]> Signed-off-by: Eduardo Barretto <[email protected]> Co-authored-by: Oliver Chang <[email protected]>
Signed-off-by: Oliver Chang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.