Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is calling checkpoint() with no args on shutdown correct behaviour? #208

Open
msinto93 opened this issue Mar 7, 2023 · 0 comments
Open

Comments

@msinto93
Copy link

msinto93 commented Mar 7, 2023

In the sample code, when a shutdown is requested we call the checkpoint() method with no args, which means the checkpoint sequence number will be the end of the most recently delivered list of records.

def shutdown_requested(self, shutdown_requested_input):
        self.log("Shutdown has been requested, checkpointing.")
        shutdown_requested_input.checkpointer.checkpoint()

Is this behaviour correct? What if shutdown is requested while we are halfway through processing a batch of records? This would mean we checkpoint the last record and therefore skip the second half of the batch. Or are we saying that shutdown_requested() can only be called after process_records() has completed, and not during?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant