Skip to content

completed openapi web search project server. #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

priyanshu-kun
Copy link
Contributor

Successfully finalized the server-side of the OpenAPI web search project. Developed an indexing endpoint, implemented indexing functionality, and incorporated efficient retrieval capabilities for search functionality.

@@ -1,6 +1,11 @@
const CC_SERVER_URL = 'https://index.commoncrawl.org/';
const RABBIT_MQ_URL = 'amqp://localhost:5672';
const ELASTIC_LOCAL_URL = 'https://localhost:9200';
const ELASTIC_HTTP_CRED = '/home/priyanshu/Downloads/ELK/primary-node/config/certs/http_ca.crt';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@priyanshu-kun remove/edit this path to be something generic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's pretty difficult to determined the location of http_ca.crt file because it varied system to system so used has to set that accordingly.

const QUEUE_NAME = 'index-files-jobs';
const ELASTIC_USERNAME = 'elastic';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please dont store usernames and passwords

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they didn't make any difference because these credentials are for local elasticsearch instance and if someone somehow get them in that case they are completely usless. But for production environment I have to rewrite the elasticseach connection driver altogether according to docs.

@HimanshuS129
Copy link
Collaborator

One suggestion, swagger validator seems to be filtering alot, I think it will also filter multi file schemas.
We should also index non-validated defintions also. WDYT @vinitshahdeo ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants