Skip to content

feat: gracefully shutdown endpoint by revoking etcd lease + python binding #730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Apr 18, 2025
Merged

feat: gracefully shutdown endpoint by revoking etcd lease + python binding #730

merged 7 commits into from
Apr 18, 2025

Conversation

tedzhouhk
Copy link
Contributor

@tedzhouhk tedzhouhk commented Apr 17, 2025 •
edited
Loading

Endpoint will gracefully shutdown (graceful = finish all generation TCP stream) after a (child) etcd lease is revoked.
Add this feature to the python binding example.

This PR is based on this commit from @ryanolson . Thanks for the prototype!

@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented Apr 17, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@tedzhouhk tedzhouhk changed the title feat: endpoint gracefully shutdown + python binding feat: gracefully shutdown endpoint by revoking etcd lease + python binding Apr 17, 2025
ishandhanani
ishandhanani approved these changes Apr 18, 2025
View reviewed changes
Copy link
Contributor

@ishandhanani ishandhanani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - another rust convert I see :)

alec-flowers
alec-flowers approved these changes Apr 18, 2025
View reviewed changes
Copy link
Contributor

@alec-flowers alec-flowers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Pls address comments before merging.

@tedzhouhk tedzhouhk enabled auto-merge (squash) April 18, 2025 18:14
@tedzhouhk tedzhouhk merged commit 4c38680 into main Apr 18, 2025
8 checks passed
@tedzhouhk tedzhouhk deleted the hzhou/grace_shutdown branch April 18, 2025 18:36
@ishandhanani ishandhanani mentioned this pull request Apr 18, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@grahamking grahamking grahamking left review comments

@alec-flowers alec-flowers alec-flowers approved these changes

@ishandhanani ishandhanani ishandhanani approved these changes

@nnshah1 nnshah1 Awaiting requested review from nnshah1 nnshah1 is a code owner

@tanmayv25 tanmayv25 Awaiting requested review from tanmayv25 tanmayv25 is a code owner

@piotrm-nvidia piotrm-nvidia Awaiting requested review from piotrm-nvidia piotrm-nvidia is a code owner

@ptarasiewiczNV ptarasiewiczNV Awaiting requested review from ptarasiewiczNV ptarasiewiczNV is a code owner

@ryanolson ryanolson Awaiting requested review from ryanolson ryanolson is a code owner

@paulhendricks paulhendricks Awaiting requested review from paulhendricks paulhendricks is a code owner

@biswapanda biswapanda Awaiting requested review from biswapanda biswapanda is a code owner

@tmonty12 tmonty12 Awaiting requested review from tmonty12 tmonty12 is a code owner

@GuanLuo GuanLuo Awaiting requested review from GuanLuo GuanLuo is a code owner

@rmccorm4 rmccorm4 Awaiting requested review from rmccorm4 rmccorm4 is a code owner

@kkranen kkranen Awaiting requested review from kkranen kkranen is a code owner

@hutm hutm Awaiting requested review from hutm hutm is a code owner

@julienmancuso julienmancuso Awaiting requested review from julienmancuso julienmancuso is a code owner

@hhzhang16 hhzhang16 Awaiting requested review from hhzhang16 hhzhang16 is a code owner

@sshchoi sshchoi Awaiting requested review from sshchoi sshchoi is a code owner

@PeaBrane PeaBrane Awaiting requested review from PeaBrane PeaBrane is a code owner

@mohammedabdulwahhab mohammedabdulwahhab Awaiting requested review from mohammedabdulwahhab mohammedabdulwahhab is a code owner

Assignees
No one assigned
Labels
size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@tedzhouhk @grahamking @alec-flowers @ishandhanani