Spike: refactor ingest sdk packaging, demarcate purpose

[sreuland]

What problem does your feature solve?

There is no clear definition in terms of single public facing ingest sdk package provided from this repo, nor are there any relevant releases of an ingestion sdk to inform otherwise, there was a release 3 years ago, stellar-ingest-v2.0.0, but much has evolved since.

The stellar-go/ingest package alone is not sufficient to write an ingestion application, various other packages in the go repo are needed and users will discover this mostly on their own, such as the Building pipeline example application which highlights several different packages from the repo which are needed.

What would you like to see?

• Investigate the feasability for a single package hierarchy/layout in this repo which demarcates a public facing sdk encompassing the functionality needed for ingestion by an application.
• Propose the new package layout with some criteria:
  • it should demonstrate application ingestion use cases
  • it should demonstrate compatibility in CI for automated build/release packaging

Some key components which likely will need to be addressed in the design for inclusion/exclusion:

1. ingest/ledgerbackend/captive_core - should this be published, or where to relocate it, something more internal for just horizon and galexie use?, etc.
2. ingest/processors and services/horizon/internal/ingest/processors, can these be consolidated..under horizon internal, they seem heavily concerned with deriving new models from network data, beyond parsing, tbd.
3. support/contractevents
4. XDrill functions, new, in development, going under stellar-go/ingest
5. Change Compactor, should it move to servcies/horizon/internal
6. others ?### What alternatives are there?

depends on: #5525 - this refactors CDP packaging, the ingest sdk should take this proposal into account, see where conflict or complementary.