Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Linux data source to Homebrew/homebrew-core #569

Merged
merged 20 commits into from
Nov 1, 2021

Conversation

issyl0
Copy link
Member

@issyl0 issyl0 commented Oct 23, 2021

  • Very WIP, opening this as very draft to a) chart my progress and b) give visibility to others who might want to do parts of Migrate to homebrew-core for Linux #566.
  • At the moment this is just me messing around trying to remember what I did here when I implemented Linux formulae on this site - that was two years ago! 😱

My vague plan here:

  • The low-hanging fruit, README, index page.
  • Remove formula-linux/ and see what breaks. (This is where I currently am at the time of writing this.)
  • Check we've got all the necessary Linux things on the macOS (I should say "formula") pages now.
  • The rest of the things in Migrate to homebrew-core for Linux #566, unless anyone else gets to them first.

Jekyll is still a mystery to me, so I can't promise this'll be fast.

@EricFromCanada
Copy link
Member

I've implemented the redirects for the HTML pages; namely, /formula-linux/ => /formula/ and /formula-linux/ƒ => /formula/ƒ. The JSON API, however, can't be properly redirected with what we have on GitHub Pages. Since redirect_from simply generates an HTML page with a meta refresh value in the old location, API users expecting to find a JSON file in the old locations will instead find HTML that they (probably) won't know what to do with.

@SMillerDev
Copy link
Member

Could we just copy the api files and send this as a feature request to github?

@MikeMcQuaid
Copy link
Member

The JSON API, however, can't be properly redirected with what we have on GitHub Pages. Since redirect_from simply generates an HTML page with a meta refresh value in the old location, API users expecting to find a JSON file in the old locations will instead find HTML that they (probably) won't know what to do with.

I think this is fine, we can just drop these files.

Could we just copy the api files and send this as a feature request to github?

Let's not. It won't be implemented in a meaningful timeframe and will bloat this repo until then.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me!

@Bo98
Copy link
Member

Bo98 commented Oct 26, 2021

Not that it blocked this or anything, but I've informed Repology that the Linux data source is now deprecated and will soon be no more.

@MikeMcQuaid
Copy link
Member

@Bo98 Thanks!

@Bo98
Copy link
Member

Bo98 commented Oct 26, 2021

Done: https://repology.org/news

@issyl0
Copy link
Member Author

issyl0 commented Oct 30, 2021

Thanks for the help, Eric, and for ticking the checklist items in the issue. There's quite a lot complete here! ✨

Over this weekend I'll work on item 2, the formula page: splitting up the views by macOS and Linux dependencies/caveats/bottles and the analytics tables.

@issyl0
Copy link
Member Author

issyl0 commented Oct 30, 2021

On the "show macOS and Linux analytics in separate tables", I'd quite like to show them in one table with multiple columns inside for macOS and Linux, but I couldn't work out how to do it with all the loops and different paths for analytics and analytics-linux. Maybe that can be a future goal!

@EricFromCanada
Copy link
Member

The issue there is how the stats table markup iterates through each formula's invocation form that appears in each data source, whether just the formula name or with options. To combine the tables would require creating a merged list of these from each data source before fetching the values for each form, if they exist. But, since in core the only possibilities are either [formula] or [formula] --HEAD, the loop could check for each form on every loop and only print the row if a value was present in either data source.

@EricFromCanada
Copy link
Member

With for loops implemented and analytics-linux added, that's a wrap for the JSON templates. Although the same kind of looping could be added to the formula HTML template, it probably needs a complete redesign.

@issyl0
Copy link
Member Author

issyl0 commented Oct 31, 2021

  1. the Linux/macOS dependencies, analytics (and perhaps: caveats, anything else?) should be separated when they differ between OSs (but still displayed on the same page). This requires updating the formula API to differentiate between macOS and Linux dependencies, conflicting formulae, keg_only status, and (possibly) caveat text.

This is the last thing to do here, and it feels quite big, and this PR is extremely large already. I know if we don't ship step 3 with steps 1, 2, 4 and 5 we'll be lacking functionality for a bit. But if people are pulling Linux data from this API now they're getting out of date stuff anyway). All that to say, does anyone else have more time than me to work on this part? @Rylan12?

@SMillerDev
Copy link
Member

Am I missing something or is the easiest solution to stop splitting Linux and macOS analytics reporting?

@EricFromCanada
Copy link
Member

EricFromCanada commented Oct 31, 2021

Am I missing something or is the easiest solution to stop splitting Linux and macOS analytics reporting?

No, that part is done. The issue now is that anything in formulae that's potentially affected by an on_linux block isn't reflected in its JSON representation, so the formula API (and therefore the formulae.brew.sh pages) are showing Mac-only info. I'd say we're good to ship what we have and follow up with more once the formula API is updated.

@issyl0 issyl0 marked this pull request as ready for review October 31, 2021 14:09
@Rylan12
Copy link
Member

Rylan12 commented Oct 31, 2021

This is the last thing to do here, and it feels quite big, and this PR is extremely large already. I know if we don't ship step 3 with steps 1, 2, 4 and 5 we'll be lacking functionality for a bit. But if people are pulling Linux data from this API now they're getting out of date stuff anyway). All that to say, does anyone else have more time than me to work on this part? @Rylan12?

Eventually, yes, I'd be happy to work on this a bit. At the moment, though, I'm a bit swamped with HOMEBREW_INSTALL_FROM_API issues and school (midterms start again soon 😭). If I end up with some more time and start to work on it, I'll leave a comment.

@MikeMcQuaid
Copy link
Member

This is the last thing to do here, and it feels quite big, and this PR is extremely large already. I know if we don't ship step 3 with steps 1, 2, 4 and 5 we'll be lacking functionality for a bit. But if people are pulling Linux data from this API now they're getting out of date stuff anyway). All that to say, does anyone else have more time than me to work on this part? @Rylan12?

Yes, I'm fine to ship without 3 for sure.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️. Rebase this and merge whenever you get a chance (because there will be inevitable conflicts soon after otherwise).

@issyl0 issyl0 changed the title [WIP] Migrate Linux data source to Homebrew/homebrew-core Migrate Linux data source to Homebrew/homebrew-core Nov 1, 2021
@issyl0
Copy link
Member Author

issyl0 commented Nov 1, 2021

I think GitHub is confused by the sheer size of this PR. There were no conflicts when I rebased on master locally.

@issyl0 issyl0 merged commit 7607599 into Homebrew:master Nov 1, 2021
@issyl0 issyl0 deleted the unseparate-linux-formulae branch November 1, 2021 14:21
@SMillerDev
Copy link
Member

Thanks @issyl0 and @EricFromCanada ! Without your contributions it'd be impossible to keep homebrew going. You can feel good knowing that you've made the world a lot better for homebrew users around the world! 👍 🎉

@Rylan12
Copy link
Member

Rylan12 commented Nov 1, 2021

I think GitHub is confused by the sheer size of this PR. There were no conflicts when I rebased on master locally.

What I usually do to avoid problems like this is only commit the changes that are to the generation files and not the actual site files. Then, once the PR is merged, the automatic generation is triggered and the updated site is pushed as an additional commit.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 2, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants