[WIP] Bloom Filter Design Doc

[thugsatbay]

What is the context for this pull request?

• Tracking Issue: [PROPOSAL]: Introduce BloomFilter index #161
• Parent Issue: [PROPOSAL] Introduce Bloom Filter Index #176
• Dependencies: NA

What changes were proposed in this pull request?

Design doc for Bloom Filter

Does this PR introduce any user-facing change?

NA

How was this patch tested?

Markdown IntelliJ plugin

[andrei-ionescu]

@thugsatbay The filename starts with a space docs/design/ 161-bloom-filter.md. I mean, before 161 is a space that shouldn't be there.

[sezruby]

Could you create PROPOSAL issue instead of this PR as we discussed in #176?

And please address the followings in the design:

• new hyperspace APIs considering new index types.
  • e.g, hs.createIndex(df, IndexConfig("indexName", indexedColumns, includedColumns")) for covering index, then what would be good APIs for bloomFilters?
  • please consider all other APIs (refresh, .. etc) and provide initial suggestions. Maybe createIndex/refreshIndex are enough (but not sure)
• refactoring plan of Actions - CreateAction/RefreshAction/OptimizeAction/...

We might need to refactor the existing code base first to support other index types.
So it would be good to have a separate document like "Refactoring to support other index types".

[thugsatbay]

@sezruby Yes sure I was going to add info regarding diff ops on index, had just put the initial work out there. Thanks for the feedback. For the refactoring I am not sure what files will eventually we touch though I have explained the major basis as internally hyperspace only is aware of covering Index and when we introduce BF that will change many places in our codebase. I think I can explain this better. Thanks, will do. :)

@andrei-ionescu my bad don't know how the space creeped in. :) Thanks

[thugsatbay]

@sezruby so should I create a new Issue for the design proposal, don't you think linking to the original issue of introducing a BF will be better ? Or I create a new issue and link #161 as parent issue ?

[rapoth]

@thugsatbay Please see this #340. Let's move away from PRs since it is making reviewing designs hard.

[thugsatbay]

#341