disutil commands fails for multiple tests

[bspradling]

Describe the bug
While running multiple tests trying to use disutil to copy files between directories, every test after the first will fail.

Traceback (most recent call last):
  File "tests/plugins/pyfakefs_test.py", line 29, in test_file_copied_again
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dir_util.py", line 163, in copy_tree
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dir_util.py", line 167, in copy_tree
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/file_util.py", line 110, in copy_file
DistutilsFileError: can't copy './test2/subdir/1.txt': doesn't exist or not a regular file

When running this tests in isolation (one at a time), they both succeed.

How To Reproduce
Please provide a unit test or a minimal code snippet that reproduces the
problem.

import unittest

from distutils.dir_util import copy_tree, remove_tree
from pyfakefs.fake_filesystem_unittest import TestCase


class DataLossPluginTests(TestCase):

    def setUp(self):
        self.setUpPyfakefs()
        self.bootstrap_fake_file_system(self)

    def test_file_copied(self):
        copy_tree("./test2/", "./test/")
        remove_tree("./test2/")

        self.assertTrue(os.path.isfile('./test/subdir/1.txt'))
        self.assertFalse(os.path.isdir('./test2/'))

    def test_file_copied_again(self):

        self.assertTrue(os.path.isfile('./test2/subdir/1.txt'))

        copy_tree("./test2/", "./test/")
        remove_tree("./test2/")

        self.assertTrue(os.path.isfile('./test/subdir/1.txt'))
        self.assertFalse(os.path.isdir('./test2/'))

    @staticmethod
    def bootstrap_fake_file_system(self):
        self.fs.create_dir("./test/subdir/")
        self.fs.create_dir("./test/subdir2/")
        self.fs.create_dir("./test2/subdir/")

        self.fs.create_file("./test2/subdir/1.txt")


if __name__ == '__main__':
    unittest.main()

Your enviroment
Please run the following and paste the output.

python -c "import platform; print(platform.platform())"
`Darwin-19.0.0-x86_64-i386-64bit`
python -c "import sys; print('Python', sys.version)"
`('Python', '2.7.15 (default, Sep 18 2018, 20:30:39) \n[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.10.43.1)]')`
python -c "from pyfakefs.fake_filesystem import __version__; print('pyfakefs', __version__)"
`('pyfakefs', '3.6.1')`

[bspradling]

Appears to be an issue with how disutil is implemented? I can't recreate it with shutil. However, shutil in 2.7 has a limitation where the src directory can not exist.

import os
import unittest
import shutil

from pyfakefs.fake_filesystem_unittest import TestCase


class DataLossPluginTests(TestCase):

    def setUp(self):
        self.setUpPyfakefs()
        # self.fs.create_dir("./test/subdir/")
        # self.fs.create_dir("./test/subdir2/")
        self.fs.create_dir("./test2/subdir/")

        self.fs.create_file("./test2/subdir/1.txt")

    def test_file_copied(self):
        shutil.copytree(src="./test2/", dst="./test/")
        shutil.rmtree("./test2/")

        self.assertTrue(os.path.isfile('./test/subdir/1.txt'))
        self.assertFalse(os.path.isdir('./test2/'))

    def test_file_copied_again(self):
        shutil.copytree(src="./test2/", dst="./test/")
        shutil.rmtree("./test2/")

        self.assertTrue(os.path.isfile('./test/subdir/1.txt'))
        self.assertFalse(os.path.isdir('./test2/'))


if __name__ == '__main__':
    unittest.main()

[mrbean-bremen]

We haven't tested distutils functionality so far - it may well be that the implementation uses OS-specific functions. I will have a look later tonight.

[bspradling]

Thanks for the quick reply @mrbean-bremen. I'm figuring this might be the case. If so, please close the issue!

[mrbean-bremen]

Hm, at a first glance I don't see anything in the code that should not work, but I can reproduce the issue. As far as I can see, there is some caching issue - the old fake os module is hold in distutils.file_util.
A workaround is to import that file in the test, e.g. do

import distutils.file_util

in your test file (even if it is not used).
I'm not sure if that can be properly fixed, but I will probably have a closer look some time later.

[mrbean-bremen]

I will probably add the import to the test code to fix the issue, similar to zipfile which had the same problem (see #427), though I will try to understand the problem better anyway - maybe there is another possibility to handle this.