Switch Win32 pipes to PIPE_WAIT with sentinel bufsize

[z2oh]

Fixes #820 . There is a lot of useful context in this issue, which I will partially reproduce below.

I have an alternate PR that fixes this problem in a less fundamental way, but without a small requirements change to libdispatch which I describe below: #853.

Problem description, copied from the aforementioned PR:

• In Use PIPE_NOWAIT on Windows to avoid blocking on writes #781, the pipe was changed from PIPE_WAIT to PIPE_NOWAIT to achieve POSIX-like O_NONBLOCK semantics, as attempting to write a large buffer would cause the dispatch queue thread to block long enough to hit a time out.
• The Windows pipe synchronization thread, _dispatch_pipe_monitor_thread, uses a blocking 0-byte read as a synchronization mechanism to signal to the actual reader threads that there is data waiting in the pipe. Of course, for this to work, the read must actually be blocking. With a PIPE_NOWAIT pipe, the Read will not block, causing the thread to spin endlessly and constantly wake the actual reader thread to perform 0 byte reads. This spinning thread is the root source of the problem in this issue.

Switching the pipe back to PIPE_WAIT potentially resurfaces the blocking writes problem, however I think that #796 (which was a follow-up PR to fix a problem with the PIPE_NOWAIT implementation) also resolves the blocking write problem in practice but not necessarily in all cases.

The real issue here is the inability to distinguish on the write side of a pipe between the following cases:

• The pipe's output buffer is full (we do not want to write as calling WriteFile will block)
• The read side of the pipe has requested a full buffer's worth of data (we do want to write because we need to make progress and there is a reader waiting for it)

This PR implements the same switch back to PIPE_WAIT as in #853, but uses a trick to enable WriteQuotaAvailable == 1 to be used as a sentinel value to distinguish the "output buffer is full" case. By always leaving a free byte of space in the buffer when writing, the only way that WriteQuotaAvailable == 0 can happen is if a reader has requested a full buffer's worth of data.

This means that when we see WriteQuotaAvailable == 1, we know the output buffer is "full" and not attempt to perform a blocking write.

The downside to this approach is that we can no longer serially write-then-read a full buffer's worth of data, because the write will not finish writing the last byte until reading has commenced. I don't see this scenario ever arising in practice: if you are trying to move data serially from one buffer to another on the same thread, there are better ways to do that than through a libdispatch pipe. But regardless, this requirement is implicit due to the serial nature of the dispatch_io_pipe tests. If if is acceptable to relax this requirement from "writes of <=bufSize are guaranteed to complete in the absence of a reader" to "writes of <bufSize are guaranteed to complete in the absence of a reader" then I think this solution is more robust than just #853.

It's entirely possible I'm missing some good reason why this requirement is there though, in which case I think #853 should be acceptable.

cc @compnerd

[tristanlabelle]

I'm comfortable with this fix

[maruel]

Thanks for looking at this! Just a comment, no opinion on the change.

[compnerd]

@swift-ci please test

[compnerd]

@swift-ci please test Windows platform