ENH: Tutorial - Python Packaging - How to make your code installable

[lwasser]

This is the second round of review for lesson 2. it was previously called "pip installable" but now is called make your code installable.

Comments are welcome through 8 January 2024 given it's a second review! Merge goal: 9 jan 2024.

This PR is the followup to this one here.

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1206246014382509
  • https://app.asana.com/0/0/1206246014367549

[Zeitsperre]

Looking good!

[Zeitsperre]

Suggested change

	For a more in-depth look at the `pyproject.toml` conventions, check out the Python Enhancement Proposal (PEP) definitions ([PEP 517](https://peps.python.org/pep-0517/), [PEP 621](https://peps.python.org/pep-0621/))

[lwasser]

@Zeitsperre - i may not include this fix. Generally, folks have suggested we avoid direct links to PEP definitions in our guidebook as they are not written in a way intended for general public consumption - especially for beginners. i think there are some pages that are more user friendly adaptations but i can't seem to remember right now where those are in the Python broader website.

i'm very open to links just maybe not directly to the Peps. we do have a page on the pyproject.toml file in our guide and a tutorial coming as well on this specific topic!

[Zeitsperre]

That's reasonable! I agree, it's a bit much for a beginner to read into. Adding it to the page that focuses specifically on that would be good.

There are times when it's nice to know what is and isn't allowed in the pyproject.toml standard, but that's really getting into the details.

[lwasser]

of course! thanks again for all of the great feedback here!

[Zeitsperre]

Git tags are one approach, but I've also seen (and implemented) versioning based on a string found within the code (good for non-VC packages, if that's your style). Is one approach being endorsed over another?

[lwasser]

Ahhh good question.

this is our page on versioning.

i actually have only done either VCS based OR using a tool such as bumpversion etc to manage versions (all with semver). We did decide to use semver as a recommendation and VCS as a default. i wonder if it's worth a small breakout mentioning that there are other ways to do this and provide a link if you know of a good one? do you think the way you've done things outside of VCS is a beginner friendly? if not, it may be the case that we want to stick with VCS for now BUT we can mention there are other options in a very brief statement with a link if that feels relevant for beginners!

[Zeitsperre]

The versioning page covers more than a few use cases that I wasn't aware of at all. I wouldn't split from SemVer (in fact, I'm doing a lot of work right now to ensure that we follow SemVer v2.0 more closely), but I'd showcase how to use bump-my-version (pyproject.toml-compatible and maintained fork of bumpversion).

My organization's projects have progressively adopted flit and it doesn't come with any tools for bumping versions (like hatch or setuptools_scm). This works for us since we often want to update version strings in different parts of the code base (e.g. hard-coded version strings in the docs to not have to import the library when generating the docs; README files, test suite, etc.), or instances where we don't want to have to change much for libraries that are built with completely different backends (like scitkit-build-core for a few C++/Python projects).

I could try modifying the Versioning page to better convey how this approach can be beneficial for intermediate-complexity projects, but it might be good to mention that there are lots of tools and approaches to performing this. Does this make sense?

[Midnighter]

Added a number of inline comments.

Overall, the guide seems clear to follow to me and highlights important choices. I was wondering about one layout/style choice: There are a lot of inline links to external guides and material. Some of them may be quite complex and are more background information. Inline links always invite me to click on them directly, so I was wondering if you think footnotes for background material could be less in your face? Certainly other people will react differently.

[Midnighter]

I wonder if the figure should contain a line like "Example Dependencies:" or similar above matplotlib and the other packages?

[lwasser]

@Midnighter Do you mean something that communicate's

"this will install both your-package and your-package's dependencies" into your python environment?

so perhaps below "your-package i could write your package's dependencies

[Midnighter]

I'm not super clear on my vision. I was just trying to put myself in a beginner's shoes and was wondering, why are those packages listed there? I might have never heard of them before. So I thought how can we communicate that they are examples of package dependencies?

[Midnighter]

I don't understand the comment, why would you name your package differently when publishing to test PyPI? Also, this guide has so far avoided mentioning PyPI, so that might cause even more confusion.

[lwasser]

Ahhhh ok you are not the first person to ask about this. Here is why. pyospackage is already on test and real pypi as i'm using it to write the lessons. if someone comes in and tries to copy the lesson exactly (some people will do that) then when they push to pypi it will be rejected as the name already exists.

https://test.pypi.org/project/pyosPackage/

so i need some way to let people know that they need to give their test package another name if they are following the tutorial exactly. but this statement has confused a few reviewers so i understand why you're asking it. i am just not sure how to make this more clear!

i did add pypi at the top of the lesson - and we will have the list of lessons on the side bar where publish to pypi is one of them!!

[Midnighter]

Ahh, that makes sense. Thank you 🙂.

[Midnighter]

An alternative that does not require git is to install from a zip of the project:

pip install https://github.com/user/repo/archive/branch.zip

[lwasser]

oh that's a lot cleaner. i've never actually done it that way! @Midnighter would that work in other applications such as:

• a requirements file
• a pyproject.toml file or
• a conda yml file?

it works great for me locally - i've never used that approach.

here is what i found online about it :

The first method (ZIP archive) might be more suitable for situations where you just want to quickly install a specific snapshot of the code without needing advanced Git features.
The second method (Git URL with branch or tag) is more versatile and allows you to easily switch between versions or update to the latest changes.
Downsides:

• The ZIP archive method might be slower for larger repositories as it needs to download the entire archive.
• The Git URL method requires Git to be installed on the system.

so one downside might be for some repos - say our gemgis package that has a really HUGE 1gb repo - it would download the entire thing before installing if what i found is correct. that could be problematic for some users.

[Midnighter]

As asked on Slack, I don't follow the point about size. With the zip archive you get a snapshot of the current branch/tag of the repo. With the git command you also have to clone the repo which should be about the same size. Or maybe I'm missing something fundamental about the way pip and git interact there.

[Midnighter]

I just wanted to mention it, though. I think for the guide it's probably better to stick to the git URI. It's more common and a bit more flexible.

[lwasser]

@all-contributors please add @Midnighter for code, tutorials

[allcontributors]

@lwasser

I've put up a pull request to add @Midnighter! 🎉

[lwasser]

Overall, the guide seems clear to follow to me and highlights important choices. I was wondering about one layout/style choice: There are a lot of inline links to external guides and material. Some of them may be quite complex and are more background information. Inline links always invite me to click on them directly, so I was wondering if you think footnotes for background material could be less in your face? Certainly other people will react differently.

@Midnighter this is a really good point. we can think about this. as generally linking out from content isn't ideal just from a content perspective. @kierisi might have some thoughts on this.

i just checked and pydata_sphinx_theme does support footnotes!

i think i'll open an issue about this as we could go back and edit the guide to fix this without slowing down merging this pr tomorrow! this is really an important style question guide .

[lwasser]

We are in great shape to merge this tomorrow. i need to just fix the install from github text - we should use the https .zip approach as the syntax is simpler and will be create lower cognitive load for users to remember!

[kierisi]

Overall, the guide seems clear to follow to me and highlights important choices. I was wondering about one layout/style choice: There are a lot of inline links to external guides and material. Some of them may be quite complex and are more background information. Inline links always invite me to click on them directly, so I was wondering if you think footnotes for background material could be less in your face? Certainly other people will react differently.

@Midnighter this is a really good point. we can think about this. as generally linking out from content isn't ideal just from a content perspective. @kierisi might have some thoughts on this.

i just checked and pydata_sphinx_theme does support footnotes!

i think i'll open an issue about this as we could go back and edit the guide to fix this without slowing down merging this pr tomorrow! this is really an important style question guide .

this is a great style guide question, and I'd have to think about it more. I think there are places where footnotes could make sense, whereas in others we want the inline link. from an SEO-perspective it shouldn't matter where in the page the link is placed (unless something's changed recently).

and SEO-wise it's great to link out to other sites - this helps us build authority and credibility, and ideally in-turn leads to people linking to our content as well. while we want people on pyopensci pages as much as possible, we should definitely be linking out to reputable sources!

[lwasser]

ok y'all!! tutorial #2 is being merged now!! thank you for ALL of the great comments and discussion in this and the previous pr's. And of course we can always change things in the future (i'm sure we will tweak things after these are actually taught!) merging this NOW as CI i generally happy (except for the new page link which is always broken on all pr's (but i've submitted an issue to understand how to address that!!)