xdist munges error message if a requested path does not exist

[pombredanne]

Using code in this branch: aboutcode-org/scancode-toolkit#414 calling py.test on a non-existing path does not return an error message with xdist, it just runs no tests:

(scancode-toolkit) pombreda@COMPUTER:~/w421/scancode-toolkit$ py.test -vvs -n 2 MISSING
========================================================================== test session starts ==========================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 -- /home/pombreda/w421/scancode-toolkit/bin/python2.7
cachedir: .cache
rootdir: /home/pombreda/w421/scancode-toolkit, inifile: setup.cfg
plugins: xdist-1.15.0
[gw0] linux2 Python 2.7.6 cwd: /home/pombreda/w421/scancode-toolkit
[gw1] linux2 Python 2.7.6 cwd: /home/pombreda/w421/scancode-toolkit
[gw0] Python 2.7.6 (default, Oct 26 2016, 20:30:19)  -- [GCC 4.8.4]
[gw1] Python 2.7.6 (default, Oct 26 2016, 20:30:19)  -- [GCC 4.8.4]
gw0 [0] / gw1 [0]
scheduling tests via LoadScheduling

===================================================================== no tests ran in 0.30 seconds ======================================================================

In contrast without xdist, it does print alright: ERROR: file not found: MISSING

(scancode-toolkit) pombreda@COMPUTER:~/w421/scancode-toolkit$ py.test -vvs  MISSING
========================================================================== test session starts ==========================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 -- /home/pombreda/w421/scancode-toolkit/bin/python2.7
cachedir: .cache
rootdir: /home/pombreda/w421/scancode-toolkit, inifile: setup.cfg
plugins: xdist-1.15.0

===================================================================== no tests ran in 0.00 seconds ======================================================================
ERROR: file not found: MISSING

[pombredanne]

And with older versions I had:

(scancode-toolkit2) pombreda@COMPUTER:~/w421/scancode-toolkit2$ py.test -vvs -n 2 I_DO_NOT_EXIST
========================================================================================================================== test session starts ===========================================================================================================================
platform linux2 -- Python 2.7.6 -- py-1.4.26 -- pytest-2.7.0 -- /home/pombreda/w421/scancode-toolkit2/bin/python2.7
rootdir: /home/pombreda/w421/scancode-toolkit2, inifile: setup.cfg
plugins: xdist
[gw0] linux2 Python 2.7.6 cwd: /home/pombreda/w421/scancode-toolkit2
[gw1] linux2 Python 2.7.6 cwd: /home/pombreda/w421/scancode-toolkit2
[gw0] Python 2.7.6 (default, Oct 26 2016, 20:30:19)  -- [GCC 4.8.4]
gw0 ok / gw1 CERROR: file not found: I_DO_NOT_EXIST
gw0 [0] / gw1 CERROR: file not found: I_DO_NOT_EXIST
[gw1] Python 2.7.6 (default, Oct 26 2016, 20:30:19)  -- [GCC 4.8.4]
gw0 [0] / gw1 [0]
scheduling tests via LoadScheduling

============================================================================================================================  in 0.26 seconds ============================================================================================================================

which had the error message but not the error exit code

[pombredanne]

I get the same munging with xdist-1.15.1.dev10+ng68470d3 (e.g. HEAD)

[pombredanne]

The version that does not munge error messages (but does not return a proper error exit code) is Pypi's 1.11 so the issue was introduced somewhere between that and Pypi's 1.15

[pombredanne]

Well, the behaviour changed exactly between 1.11 and 1.12 there. 1.12 returns a proper error code BUT munges the error message.

(scancode-toolkit) pombreda@COMPUTER:~/w421/scancode-toolkit$ py.test -vvs -n 2 MISSING
========================================================================== test session starts ==========================================================================
platform linux2 -- Python 2.7.6, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 -- /home/pombreda/w421/scancode-toolkit/bin/python2.7
cachedir: .cache
rootdir: /home/pombreda/w421/scancode-toolkit, inifile: setup.cfg
plugins: xdist-1.12
[gw0] linux2 Python 2.7.6 cwd: /home/pombreda/w421/scancode-toolkit
[gw1] linux2 Python 2.7.6 cwd: /home/pombreda/w421/scancode-toolkit
[gw0] Python 2.7.6 (default, Oct 26 2016, 20:30:19)  -- [GCC 4.8.4]
[gw1] Python 2.7.6 (default, Oct 26 2016, 20:30:19)  -- [GCC 4.8.4]
gw0 [0] / gw1 [0]
scheduling tests via LoadScheduling

[RonnyPfannschmidt]

thanks for detailed investigation on when it was introduced

[RonnyPfannschmidt]

@pombredanne how exactly is that error communicated to pytest?

[pombredanne]

@RonnyPfannschmidt beats me... I would not know. At the moment this is not critical it just makes debugging this difficult because you only get an error "5" return code and no message is printed that would tell me that I asked for a path that does not exist at all

[pombredanne]

Somehow I think that this is the commit that started the munge (and started returning a proper exit code too) d84f1f0

[RonnyPfannschmidt]

ping to @nicoddemus wrt that one then :)

[nicoddemus]

Thanks guys, I will take a look when I get a chance. 👍