Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a json output format of cargo package --list #11666

Open
messense opened this issue Feb 1, 2023 · 4 comments · May be fixed by #14262
Open

Provide a json output format of cargo package --list #11666

messense opened this issue Feb 1, 2023 · 4 comments · May be fixed by #14262
Labels
C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` Command-package S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted.

Comments

@messense
Copy link

messense commented Feb 1, 2023

Problem

Currently cargo package --list outputs filenames line by line, it works fine normally. But when Cargo.toml contains metadata that uses relative path to point to parent directory, there is no way to get that information. For example readme = "../README.md:

[package]
readme = "../README.md"

cargo package --list outputs

README.md

instead of

../README.md

It makes sense because it's the filesystem layout in the .crate file, but for downstream tooling like maturin it doesn't know that the READMD.md is actually coming from ../README.md, which causes PyO3/maturin#1431

Proposed Solution

Add a json output format, for example cargo package --list=json outputs something like

{
  "README.md": "../README.md",
  "src/lib.rs": "src/lib.rs"
}

as proposed by @konstin in PyO3/maturin#1442 (comment)

Notes

No response

@messense messense added the C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` label Feb 1, 2023
@messense
Copy link
Author

messense commented Feb 3, 2023

Related to #11597, but CARGO_PKG_README isn't usable outside of Cargo build process.

cc @weihanglo

@weihanglo
Copy link
Member

Thanks for the proposal!

Although this doesn't sound too hard to implement, we should still make sure that we have thought of different workarounds and designs before moving on.

For workarounds, off the top of my head there are at least two:

  • cargo metadata | jq '.packages[] | select(.name == "<package-name>") | .readme' to get the path.
  • Make the README.md file a symlink. Cargo takes care of symlinks and copies over.

The second one is definitely a feasible option I feel like.

For the design of this proposal, I'd like to know apart from the issue from maturin, is there any other issue that this proposal can resolve? One bug could probably be fixed with this proposal: cargo package --list doesn't escape or quote filenames. If a path contains newlines it becomes hard to parse by machines. The other question is that, do people want to know more on filesize, file permission, and many more? What information should be included in that JSON output? We should also make sure that we don't push the responsibility of cargo package too far, as it is first served as “listing files being packaged”.

@konstin
Copy link
Contributor

konstin commented Feb 13, 2023

For me, this isn't specifically about the readme, but more generally about allowing an external to create a .crate like package. We need to e.g. include the readme, license files, Cargo.toml and Cargo.toml.orig. It would be really neat if cargo would provide a json interface to get everything you need for that similar to what cargo metadata currently does really nicely for the dependency tree; Basically, prepare writing the .crate, write all the generated files to the target directory and dump a path mapping as json. I don't think this needs any more information than just the path, when in doubt we can still stat the files.

While it would technically be possible to just read the .crate, that seems both rather wasteful and makes it difficult to combine with the files we need to add for python source distributions.

@messense
Copy link
Author

messense commented Feb 14, 2023

While it would technically be possible to just read the .crate

We can't if there are path dependencies without a version key, Cargo refuses to package them.

@LawnGnome LawnGnome linked a pull request Jul 17, 2024 that will close this issue
@epage epage added S-needs-team-input Status: Needs input from team on whether/how to proceed. S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. and removed S-needs-team-input Status: Needs input from team on whether/how to proceed. labels Jul 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` Command-package S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants