Connection pool optimization: reduce connection maintenance loops complexity

[MarkusSintonen]

Summary

This is a small low hanging fruit for optimizing and simplifying the connection maintenance loops in the connection pool. This work is based on suggestion by @T-256 here (thanks!). Previously performance of httpcore degraded as connection count increased in the pool. As the maintenance loops had to do more and more work iterating and reiterating the connections.

This optimization brings the performance of httpcore to same level as urllib3 with sync-usage.

The benchmarks below include the socket polling fix. (The socket polling problem makes request processing latency so highly varying that it overshadows everything.)

Previously with sync (with optimized socket polling):

PR with sync:

As it can be seen the performance gets to exact same level as urllib3. There is almost no overhead from httpcore related request processing anymore.

Checklist

• I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.

[MarkLux]

wonder if there are any further progress for this PR, our team are waiting for this fix @MarkusSintonen @T-256

[MarkusSintonen]

@MarkLux I don't know where the author has gone :/