Intermittent 503 Errors (upstream_reset_before_response_started{connection_termination}) with Istio and Uvicorn Despite 200 Status in Logs

[harshkumar314e]

I'm encountering an intermittent issue while using Uvicorn with FastAPI in a Kubernetes environment managed by Istio.
I have configured Uvicorn to run with 5 workers. Occasionally, for some requests, the Uvicorn access logs indicate a successful 200 status response, but Istio returns a 503 error to the client with the message upstream_reset_before_response_started{connection_termination}.

Environment:

• Uvicorn Version: 0.32.1
• FastAPI Version: 0.115.6
• Python Version: 3.11.9
• Istio Version: 1.20.0

Configuration:

• Uvicorn is running with 5 workers

I suspect there might be a connection issue between Istio and Uvicorn. Any insights or suggestions on how to troubleshoot or resolve this issue would be greatly appreciated.

P.S: There was not much load on the server when the 503 error occurred, and many other requests at the same time were successful. Also, this is happening at least once a day.

[harshkumar314e]

Anything I should I try or I am missing, let me know if any other detail is required.