-
Notifications
You must be signed in to change notification settings - Fork 500
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
External Search Refactor #11281
Draft
qqmyers
wants to merge
14
commits into
IQSS:develop
Choose a base branch
from
GlobalDataverseCommunityConsortium:Search_Refactor
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
External Search Refactor #11281
qqmyers
wants to merge
14
commits into
IQSS:develop
from
GlobalDataverseCommunityConsortium:Search_Refactor
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it: This is a draft PR demonstrating the ability to have several configurable search engines. It's primarily a proof-of-concept of the general design that's been discussed, but, with the example ExternalSearchServiceBean and PostExternalSearchServiceBean classes it could already support developing/demonstrating a separate tool. Those beans send all of the search parameters from Dataverse to a third party web service (via GET or POST) and expect to get back an ordered JsonArray of matching entityIds. They will then send a query to Solr via Dataverse's internal SolrSearchServiceBean that will generate the output needed for the display. (Since Solr doesn't preserve the order in the query, these beans reorder the results.)
Which engine is used is controlled by a Mprofile/jvm option: -Ddataverse.search.default-service=
where name is 'externalSearch', 'postExternalSearch', etc. as defined in the classes.
Both of the external classes also rely on a new/temporary :ExternalSearchUrl parameter (easier to change than a JvmSetting)
The ExternalSearchServiceBean has been 'tested' with :ExternalSearchUrl pointed at a static page returning a list of valid entityIds - can see in the logs that all the search params are sent as query params, can verify that final the search results contain only those entities, that the order is the same as what's sent, etc. The POST version has not been tested since there's no service setup to receive a POST.
There are other example beans: goldenOldies that only returns entries with entityIds <1000, and oddlyEnough that only returns entityIds that are odd. Neither of these is practical, but they allow some testing w/o any external service and, for oddlyEnough, show how one can deal partially deal with paging, changing the number of total results, etc. that come from solr.
Which issue(s) this PR closes:
Special notes for your reviewer:
The ability to find/enable SearchService implementations packaged in separate jars is coded but has not been tested.
If only the existing Solr engine needs to be a bean, this code could be much more like the code for PidProviders and exporters (which are known to work).
The SolrSearchServiceBean is currently a @singleton. I changed while testing other things and am not sure if it is still needed or if it can be @stateless again.
Suggestions on how to test this:
Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Is there a release notes update needed for this change?:
Additional documentation: