EPS does not notice when a connection is dead (IOError)

[micressor]

Steps to reproduce:

1. Connect electrum/bitbox to your EPS via smartphone
2. Set smartphone into flight mode
3. After 20 minutes, still no new connection can be opened.

openssl s_client -connect epsserver:50002

The timeout of the socket came only after 25 minutes

DEBUG: 16:16: IOError: TimeoutError(110, 'Connection timed out')

Workaround:

systemctl restart electrumpersonalserver

I think something with the connection management is not running ideally.
My wish is, that as long as EPS can only handle one connection, the timeout should be very short.

electrum-personal-server/electrumpersonalserver/server/common.py

Line 160 in cb6f76b

except (ConnectionRefusedError, ssl.SSLError, IOError):

versions: eps 0.2.1.1

[chris-belcher]

Thanks for the detailed issue.

Probably the best way to solve this is to have the server ping the client, there is a protocol method for pinging: https://github.com/spesmilo/electrumx/blob/master/docs/protocol-methods.rst#serverping

There is already a place in the code to add such a ping:

electrum-personal-server/electrumpersonalserver/server/common.py

Lines 198 to 207 in 6909e76

	except socket.timeout:
	poll_interval_change = mempool_sync.poll_update(1)
	if poll_interval_change == PollIntervalChange.FAST_POLLING:
	sock.settimeout(fast_connected_timeout)
	elif (poll_interval_change
	== PollIntervalChange.NORMAL_POLLING):
	sock.settimeout(normal_connected_timeout)
	on_heartbeat_connected(poll_interval_connected, rpc,
	txmonitor, protocol)

the same place where new transactions and blocks are checked for, and the same place where the mempool is queried.

If not sending a ping, those docs say that servers can disconnect clients after a timeout.

Just so everyone knows, unfortunately this issue cant be a priority for me for the next year or so. If someone else codes it I can help review though.

[micressor]

@chris-belcher Thank you for your answer. Isn't there an easier way to set the 25 minute timeout further down? Of course, implementing the ping method is the best method in the long-term.

[chris-belcher]

You're right, there might be. The TCP socket option called keepalive might do the the trick https://www.programcreek.com/python/example/4925/socket.SO_KEEPALIVE

But if you're talking about the socket.settimeout() method which is invoked a couple of times near that code, that wont be it. That kind of timeout just means the interval at which the socket stops waiting for stuff to happen and instead sends control flow somewhere else. In EPS this feature is used to query the bitcoin core node for information about new transactions, new blocks or the state of the mempool.

[chris-belcher]

I realize that the protocol method I linked only works from the client to the server, not from the server to the client.

I tried making a hanging connection to public electrum servers out there, I just made a connection and didn't send anything. The servers don't send any kind of ping and leave the connection open for a long time (I gave up wait after about 30 minutes).

So this issue doesn't have an easy fix. The fundamental problem comes from only supporting one connection at a time, so it will be fixed when issue #50 is implemented.