Flush streams

[m-ou-se]

Python uses the stdout and stderr FILE*s from libc, which by default only flush on \n when the output is a tty.

This means that when the output is not a tty, output can be lost when Rust code doesn't flush the C streams before exiting, such as in a panic.

This explains why no error message was seen in #31 at first.

[m-ou-se]

Unfortunately, the libc crate doesn't expose the stdout and stderr FILE*s. :(

[m-ou-se]

Just flushing stdout/stderr using fflush() doesn't seem to work. Seems like CPython does its own buffering.

Adding sys.stdout.flush() in Python does work.

[m-ou-se]

Looks like CPython calls sys.stdout.flush() and sys.stderrr.flush() internally during cleanup: https://github.com/python/cpython/blob/1cba1c9abadf76f458ecf883a48515aa3b534dbd/Python/pylifecycle.c#L1204-L1232

But it doesn't seem to expose a function that does that doesn't exit/cleanup everything.

[m-ou-se]

PyO3 also tried to work around this problem, by simply executing import sys; sys.stderr.flush() before panicking:

https://github.com/PyO3/pyo3/blob/072be6ce8376edae60963bf0e6db7c32a28e144d/src/lib.rs#L304-L305

[m-ou-se]

Seems like the right thing to do is to call Py_Finalize before exiting. That'll flush the streams, join threads, etc. But running that function with atexit() results in a segfault.

[m-ou-se]

Segfault doesn't occur when acquiring the GIL first :)

Opened a PR on PyO3: PyO3/pyo3#943