open is not skipped for skipped modules

[giladreti]

While experimenting with the package, I have found that the open builtin is faked for even for modules that I have excluded from faking via the Patcher context manager.

Traceback (most recent call last):
  File "c:\Users\gilad\Documents\my_module\test.py", line 9, in <module>
    test()
  File "c:\Users\gilad\Documents\my_module\test.py", line 7, in test
    my_func()
  File "c:\Users\gilad\Documents\my_module\my_file.py", line 7, in my_func
    open('test.py')
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 4449, in open
    return fake_open(file, mode, buffering, encoding, errors,
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 4935, in __call__
    return self.call(*args, **kwargs)
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 4983, in call
    file_object = self._init_file_object(file_object,
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 5037, in _init_file_object
    self.filesystem.raise_os_error(errno.ENOENT, file_path)
  File "C:\Users\gilad\AppData\Local\Programs\Python\Python38\lib\site-packages\pyfakefs\fake_filesystem.py", line 960, in raise_os_error
    raise OSError(errno, message, filename)
FileNotFoundError: [Errno 2] No such file or directory in the fake filesystem: 'test.py'

How To Reproduce
I have the following setup:

.
├── __init__.py
├── __pycache__
│   ├── __init__.cpython-38.pyc
│   ├── my_file.cpython-38.pyc
│   └── test.cpython-38-pytest-6.0.2.pyc
├── my_file.py
└── test.py

test.py:

from pyfakefs.fake_filesystem_unittest import Patcher

from my_file import my_func

def test():
    with Patcher(additional_skip_names=['my_file']):
        my_func()

test()

my_file.py:

import os

def my_func():
    os.open('test.py', os.O_RDONLY, 0o777) # works!

    open('test.py') # fails...

Your enviroment

python -c "import platform; print(platform.platform())"
python -c "import sys; print('Python', sys.version)"
python -c "from pyfakefs.fake_filesystem import __version__; print('pyfakefs', __version__)"

The output is as follows:

Windows-10-10.0.19041-SP0
Python 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)]
pyfakefs 4.1.0

Could a possible solution be patching the builtin open by overriding it only in the modules not listed for skip? It should be checked that the module hadn't overrode the open by itself, since it may cause overriding it again.

[mrbean-bremen]

Thanks for the report -- this has probably never been used so far, so it has not been noticed.

[mrbean-bremen]

This one may be a bit tricky, because other than all other patched imports, open is imported implicitely from buildins, so the usual mechanism does not work here. I have to think about this...

[mrbean-bremen]

Out of interest - what is the use case you need this for?

[giladreti]

I am using the botocore library, which generates objects dynamically based on data files that come with it. It uses the builtin open for that, and mocking the fs making it break as it cannot find its config files. As a temporary solution I am using add_real_directory for adding the necessary files to the fake fs, but this is not ideal...

[mrbean-bremen]

As a temporary solution I am using add_real_directory for adding the necessary files to the fake fs, but this is not ideal...

Actually I used to do the same for some django-related package (I think tzinfo, which loads the timezone info from disk). I agree that this is not ideal (though it works well enough, and has no real performance penalty), so to fix these things is a good idea.

[hard7]

Hi. In my project I have a file logsio.py and it has open inside. So pyfakefs doesn't fake it. I tried to do pyfakefs.fake_filesystem_unittest.Patcher.SKIPNAMES -= {"io"} but it doesn't look safe and fs.add_real_file doesn't work well. Changing the source name or moving open to another source file should solve the issue but I'd like to fix that on the pyfakefs side (without changes in repo). Do you know how?

[mrbean-bremen]

@hard7 - thanks for the report--this is obviously a bug. logsio shall not get skipped just because it ends with io. Can you please write a new issue for this, as it is not related to this one (other than it may have been introduced while fixing this one)? I will try to fix this ASAP.