Added code references to search results.

[sarahboyce]

Description

Recently, we added a scroll to text fragment to improve the search result navigation experience.
This takes the first line that was found in the body and scrolls you to this line.

This PR adds references to our Python class/class method/function definitions which will appear as a sub-item of the page search result if they match the search.
This means that when someone searches for "select" they have choice as to what to navigate to. They can chose the "QuerySet.select_related" or "QuerySet.select_for_update". The anchors of these are also robust and so you will navigate to the definition of these (the scroll for text logic is a little fragile due to the html stripping logic of full text search).

This "choice" is even more useful when searching for something like "field":

---

Issues aiming to fix

Ideally this along with the previous highlight text change fixes #1649 and fixes #1734

Review notes

Note that I have added some folks as possible reviewers.
I am not 100% sure the review process given the new website WG but would appreciate feedback as I'd love to land the PR 🫶

This PR has 2 commits, the first commit is a refactor commit to split the test file into multiple test files, the second is the feature.

I originally used regex to full out these code references. @bmispelon rightly pointed out this is brittle. I have since written a custom Sphinx builder in order to use Sphinx's own functionality for getting the python objects.

[alexgmin]

Is there a case in which it makes sense to have the first line of the code reference?
Based on my searches it always seems redundant with the second line

Everything else looks good to me.

[sarahboyce]

Is there a case in which it makes sense to have the first line of the code reference?

It's true that it's duplicate information.
I personally thought reading QuerySet.select_related is nicer than reading django.db.models.query.QuerySet.select_related, but still kept the full path as it's interesting/useful 🤷

[sarahboyce]

I could make it like:

QuerySet.select_related
django.db.models.query

[alexgmin]

It's true that it's duplicate information. I personally thought reading QuerySet.select_related is nicer than reading django.db.models.query.QuerySet.select_related, but still kept the full path as it's interesting/useful 🤷

It is, and I would prefer to keep the short version, but I don't know if there's a case in which the short version is not unique.

I could make it like:

QuerySet.select_related
django.db.models.query

This could also work better than the current one if only the shortened version is not viable due to uniqueness.

[sarahboyce]

It is, and I would prefer to keep the short version, but I don't know if there's a case in which the short version is not unique.

Field for example, django.db.models.fields.Field and django.forms.fields.Field

[sarahboyce]

Could we merge the test module restructuring first? It seems like a good change, and it would make reviewing this easier.

#1965

[bmispelon]

This is shaping up to be amazing, I can't wait for it to be running in production.

It looks great overall, I haven't found anything that seems wrong. I've left a few ideas for improvements, mostly around readability and trying to name things in a way that helps understand what's going on. Don't hesitate to push back on my suggestions if you think they don't make sense to you: at this point you're the domain expert here.

At the risk of repeating myself: amazing work! ✨

[bmispelon]

Could we merge the test module restructuring first? It seems like a good change, and it would make reviewing this easier.

#1965

I took the liberty of merging that PR and rebasing + force-pushing this one. It made my reviewing easier and I hope I haven't messed up your local branch too hard 😅

[bmispelon]

I think the confoverrides["extensions"] is not needed (see comment), otherwise I think this is ready to ship as far as I'm concerned 👍🏻

[bmispelon]

The double instantiating of Sphinx() was clever! I've made an alternate proposal that uses sphinx.config.Config() rather than a whole Sphinx application.

I also noticed that the multi-cpu processing had been lost, so I left a suggestion for how to re-add it. I think the logic around the conditional verbosity has also not been carried over, but I don't think that's a huge deal.

We're almost there! 😁

[bmispelon]

Fantastic work!

From my perspective it's ready to deploy, but I'll probably wait from an official 👍🏻 from the website WG before I do so.