Incorrect pyc cache loaded when test file is updated in short time

[jnhyperion]

given this test:

def test_example(pytester):
    pytester.makepyfile("""
    def test_dummy1():
        def func():
            pass
        print(func.__qualname__)
    """)
    r = pytester.runpytest("-s")
    assert r.ret.value == 0
    assert "test_example.py test_dummy1.<locals>.func" in r.stdout.str()
    pytester.makepyfile("""
    def test_dummy2():
        def func():
            pass
        print(func.__qualname__)
    """)
    r = pytester.runpytest("-s")
    assert r.ret.value == 0
    assert "test_example.py test_dummy2.<locals>.func" in r.stdout.str()

the test file is modified and reload in short time, when the 2nd test execution start, it will still load the test file before modified from __pycache__, so the test will randomly failed.