Remove Whitelist - Replace with Target and Seeds / allow Target Seeds distance-1

[liquidsec]

When a whitelist is defined, it explicitly dictates what is considered in-scope for the scan, as expected.

However, any items then in the target list that are not also present in the whitelist are considered out-of-scope. Targets not in the whitelist are ignored, which includes any DNS discovery processes.

This is not expected, considering the definitions of these items. I would expect the anything defined in the targets to still be used for DNS discovery purposes.

 -t TARGET [TARGET ...], --targets TARGET [TARGET ...]
                        Targets to seed the scan
  -w WHITELIST [WHITELIST ...], --whitelist WHITELIST [WHITELIST ...]
                        What's considered in-scope (by default it's the same as --targets)
  -b BLACKLIST [BLACKLIST ...], --blacklist BLACKLIST [BLACKLIST ...]
                        Don't touch these things
  --strict-scope        Don't consider subdomains of target/whitelist to be in-scope

The concept of seeding is currently very limited in this situation.

If i define IPs only in my whitelist, and then define a domain for seeding, currently nothing will happen. The domain will be considered out of scope, no subdomain discovery will occur for it.

Currently, there is currently little point in even having the whitelist as a separate option, because if you use it, it behaves almost exactly like a target list would.

Expected behavior:

If i define an IP based whitelist, and then seed with a domain, I expect DNS discovery to occur for that domain. I would expect any discovered items that fall into the IP ranges to be reported, while any that are discovered which do not to be ignored as out of scope.

When just a target is defined:

• Copy the target to the whitelist, as is done currently

When both a whitelist AND a target list are defined:

• The whitelist should also not be driving discovery, it should only be present to make scope decisions.
• No module should run directly against whitelist items
• Any module that does not define "in_scope_only" should still by run against the target
• DNS discovery should still occur against the target without limitations, aside from the blacklist

When only a whitelist is defined:

• Abort the scan and issue an error message stating a target must be defined

A partial workaround for the above scenario is to put the domain in the whitelist and turn on --strict-scope. However this has two flaws:

• The target becomes fully meaningless at this point
• If the base domain itself is not in scope, putting it in the whitelist may be undesirable.

[liquidsec]

Easy to reproduce example:

poetry run bbot -m dnsbrute -t blacklanternsecurity.com -w 172.64.151.232 104.18.36.24

The IPs are for blog.blacklanternsecurity.com. It does not find it.

Finds it with:

poetry run bbot -m dnsbrute -t blacklanternsecurity.com -w 172.64.151.232 104.18.36.24 -c scope.search_distance=2

[TheTechromancer]

Best solution to this is probably to get rid of the term "whitelist" and replace it with "target". Then, add a "seeds" option and allow users to override seeds instead of overriding the whitelist. This would also slightly change the functionality in that if seeds weren't specified, they would be copied from the whitelist instead of the other way around.

Also, passive enumeration modules should be modified to accept distance-1 seeds.

[liquidsec]

Im gonna update this issue to reflect the agreed upon solution