[pgmq-python] Merge pgmq-sqlalchemy with current python client package

[jason810496]

Hi, after discussing with @ChuckHend,
I will merge pgmq-sqlalchemy with the current Python client package.

The process of merging these two Python packages might include:

• Migrating documentation:
  • Add RST format docs for client class methods.
  • Add documentation with example usage.
• Migrating unit tests.

Other discussions include:

• For the pgmq-sqlalchemy client (to be merged into tembo-pgmq-python):
  • Add transaction support (allow injecting db_session).
• For the current tembo-pgmq-python client:
  • Refactor with the Python DBAPI interface to support psycopg2 and psycopg3.
• Designing the package hierarchy.

The overall process might result in two client classes:
one with SQLAlchemy support and another that directly interacts with the DBAPI.

Are there any concerns that should be addressed during this migration?

[ChuckHend]

I like pgmq-sqlalchemy because it has better support for the various SQL client libraries in the python ecosystem, and sync/async. tembo-pgmq-python only supports psycopg3 at the moment, and I think most of industry still uses psycopg2. It would be much simpler to have a single recognized client per language, so I am all for merging the two projects.

I will merge pgmq-sqlalchemy with the current Python client package.

That links to tembo-pgmq-python, but what we are discussing here is merging pgmq-sqlalachemy into tembo-pgmq-python, correct?

---

From what I can tell, pgmq-sqlalchemy and have very similar APIs, e.g. pgmq.send(), pgmq.send_batch(). class init is different, since tembo-pgmq-python only handles psycopg3. @jason810496 do you see any places across the two APIs that are vastly different? If they are the same, then maybe we can get away with a single class. Otherwise, yeah two classes would be fine to start then we could aim to collapse them in a later release.

[jason810496]

There isn’t any significant difference between the two packages.

I’m just confirming the final details; then I will start the process.
After that, there will be a large PR that needs to be reviewed 🙏

Extract pgmq Functions from the Queue Class

I think we should extract all the functions from the Queue class into a func module, which will also support inflight transactions.

For example:

from tembo_pgmq_python import func 

# inflight db session 

msg_id = func.send(db_session, msg)
# other business logic 
db_session.commit()

Consolidate into One Queue Class

Additionally, the Queue class should be constructed using either the SQLAlchemy or psycopg interface.
The Queue class will also use the func module.

from . import func 

class Queue:
    # …
    def send(self, msg):
        # distinguish between SQLAlchemy or psycopg 
        func.send(db_session, msg)

The overall structure will look like this:

.
├── func    # pgmq functions extracted from the previous Queue class
├── queue 
└── schema

[ChuckHend]

@jason810496 , am I correct in assuming this will be a non-breaking change for the existing psycopg3 users? Seems like that will be true but I am not sure.

[jason810496]

Yes, they just need to change the way the client is constructed; all the methods provided by the client class remain unchanged.
The refactored version is forward-compatible. I can also provide some migration examples in the documentation.