About two Rust PURL implementations

[pombredanne]

@matt-phylum about https://github.com/package-url/packageurl.rs vs. https://github.com/phylum-dev/purl ... I would be interested to get your take on the difference between the two?

[matt-phylum]

At Phylum, we wanted to transition away from a proprietary package identification scheme to PURL and we had some problems with althonos/packageurl.rs (aka packageurl/packageurl.rs).

@criminosis (a coworker) found that slashes were not being encoded the same as in package-url/packageurl-java, which was causing problems for us because we were using both implementations, and so he created althonos/packageurl.rs#6, although it turns out the real bug was package-url/packageurl-java#122. We ended up using a fork of althonos/packageurl.rs for a while so it would be consistent with the (broken) package-url/packageurl-java because of the lack of activity in the althonos/packageurl.rs repository.

At the same time, we were having problems with multiple implementations of our own proprietary package identification scheme and inconsistent name normalization that was causing alignment problems with data from different sources, especially PyPI, so we were really focusing on getting the name normalization correct.

We decided rather than trying to take over maintainership of althonos/packageurl.rs we would create our own implementation and transition towards using that. I think there may have been some specific reasons why we started a new implementation instead of trying to revive the old one, but I don't remember them.

Compared to althonos/packageurl.rs:

• The GenericPurl type knows nothing about package types. It is only concerned with parsing and unparsing the structure of a PURL, and has a pluggable system for package type support (see also Concerns with type-specific component value transformations #38). This allows us to be more careful about how the package types behave, and also allows anybody else who needs different behavior to get that behavior without making a completely independent implementation.
• The optional Purl type (alias of GenericPurl<PackageType>) does know about package types but is more careful about correctness to the ecosystem than correctness to the PURL spec, and only supports types which we have verified:
  • NuGet names are converted to lowercase: NuGet package names should not be case sensitive #226. pkg:nuget/newtonsoft.json and pkg:nuget/Newtonsoft.Json and pkg:nuget/Newtonsoft.JSON are all the same package and any data associated with that package needs to be accessible for any of those names.
  • PyPI names are normalized correctly: Add more specific details around normalised PyPI package naming per PEP 503 #165. It's basically the same problem as with NuGet. pkg:pypi/ruamel.yaml and pkg:pypi/ruamel-yaml and pkg:pypi/ruamel..........yaml are all the same package.
  • NPM names are not converted to lowercase: Letter case for npm packages #136. This is the opposite problem. pkg:npm/jquery and pkg:npm/jQuery are not the same package and data associated with one should not be returned for the other.
  • Go names are not converted to lowercase: Go suggestions for PURL-TYPES.rst #196. This is similar to the NPM problem. I don't know of any examples of different capitalization resulting in a different module, and it's unlikely because of the way Go modules are typically hosted, but it is possible, and Go's native tooling does care that the capitalization is correct (going so far as to come up with a special escaping scheme to be compatible with case-insensitive file systems), so lowercasing the names can be a one-way, data destroying operation that complicates or prevents getting the expected module again later.
  • During formatting, + is escaped as %2B: define handling of plus and space #261. This one isn't as important to us, but it avoids problems for our users if they are taking PURLs from our output and providing those PURLs to another product which uses a different implementation of PURL with a different meaning of +.