feat: Add DeleteEntitiesIter for viur-transactionmarker

[ArneGudermann]

No description provided.

[sveneberth]

This time is too short.

If you delete this before the task run the task would be abortet.

viur-core/src/viur/core/tasks.py

Lines 215 to 218 in b617b71

	marker = db.Get(db.Key("viur-transactionmarker", env["transactionMarker"]))
	if not marker:
	logging.info(f"""Dropping task, transaction {env["transactionMarker"]} did not apply""")
	return

We need a delta of a month or something like that because you can delay tasks in the future and tasks may fail a few times.

[ArneGudermann]

Okay I set this to one month

[ArneGudermann]

Okay the problem is the max countdown for a Transaction is 270 seconds and only 90 in Idle so we can delete this after 270 seconds imo.

[phorward]

Hi @ArneGudermann, can you comment this thing in the source-code? A reference to this issue would also be well.

[sveneberth]

Yes, the transaction times are correct. But they are totally irrelevant for the transaction marker.

The reason is that a deferred task was called within a transaction:

viur-core/src/viur/core/tasks.py

Lines 574 to 578 in fa3f977

	if db.IsInTransaction():
	# We have to ensure transaction guarantees for that task also
	env["transactionMarker"] = db.acquireTransactionSuccessMarker()
	# We move that task at least 90 seconds into the future so the transaction has time to settle
	_countdown = max(90, _countdown) # Countdown can be set to None

And the tasks should only run if the transaction was successful and that is when the marker entity exists. If the transaction fails, the marker is also removed in the rollback.
AFTER the transaction has been successful, this marker object must now exist at least until the task has been fully executed. And when the task is executed can be at the earliest 90 seconds after the call, but also any time in the future. The task can be called with an eta for the next month or a countdown of 90 hours. But the task queue could also be so full that the task is simply only executed after thousands of others. There is no fixed time here. You can't predict that. Actually, the transaction marker can therefore only be safely removed after the task has been fully executed and not at any fixed date. In reality, however, tasks are completed within 30 days of the call deferred call and anything after that can be considered a corpse IMO.

If you remove the markers BEFORE the task has been executed, this would lead to the assumption that the transaction was not successful. Which in turn means that the task is also not executed:

viur-core/src/viur/core/tasks.py

Lines 214 to 218 in fa3f977

	if "transactionMarker" in env:
	marker = db.Get(db.Key("viur-transactionmarker", env["transactionMarker"]))
	if not marker:
	logging.info(f"""Dropping task, transaction {env["transactionMarker"]} did not apply""")
	return

But that would be wrong.

[ArneGudermann]

Okay I think I missunderstood this Feature im the Frist Place. Thank you for the explaing. We can set the time to one month or we can create a conf variable for that. Or we can set a success to the Marker after the transaction.

[phorward]

Hi @ArneGudermann, can you comment this thing in the source-code? A reference to this issue would also be well.