pipx: python setup.py egg_info did not run successfully

[iainelder]

Bug Report

Because dvc is an application, I expect to be able to install it globally in an isolated virtual environment using pipx to avoid dependency conflicts with any other Python-based software (see #7767 for an example conflict).

I can't install dvc using pipx. It fails with a cryptic error: AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts' for any one of the dependent packages.

If I create a temporary virtual environment and install the package that it complains about there, there is no problem. Weirdly, that also solves the problem for the package in pipx, but the same error occurs for the next package.

I'm thinking this is something broken in my setup rather than a problem with DVC itself. Nonetheless I'd appreciate some help to fix this because it's blocking my adoption of DVC and I really want to learn about it :-)

Reproduce

With a clean pip cache, the installation fails on the package pygtrie.

$ pipx install dvc[s3]
Fatal error from pip prevented installation. Full pip output in file:
    /home/isme/.local/pipx/logs/cmd_2022-05-18_15.34.11_pip_errors.log

pip seemed to fail to build package:
    pygtrie>=2.3.2

Some possibly relevant errors from pip install:
    error: subprocess-exited-with-error
    AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'
    error: metadata-generation-failed

Error installing dvc from spec 'dvc[s3]'.

Content of /home/isme/.local/pipx/logs/cmd_2022-05-18_15.34.11_pip_errors.log:

PIP STDOUT
----------
Collecting dvc[s3]
  Downloading dvc-2.10.2-py3-none-any.whl (401 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 401.8/401.8 kB 3.7 MB/s eta 0:00:00
Collecting funcy>=1.14
  Downloading funcy-1.17-py2.py3-none-any.whl (33 kB)
Collecting pygtrie>=2.3.2
  Downloading pygtrie-2.4.2.tar.gz (35 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'

PIP STDERR
----------
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-dy4f7mv_/pygtrie_8a4fafe50c37499ebc5c52a2517a31a6/setup.py", line 179, in <module>
          distutils.core.setup(**kwargs)
        File "/home/isme/.local/pipx/shared/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 109, in setup
          _setup_distribution = dist = klass(attrs)
        File "/home/isme/.local/pipx/shared/lib/python3.8/site-packages/setuptools/dist.py", line 477, in __init__
          _Distribution.__init__(
        File "/home/isme/.local/pipx/shared/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 293, in __init__
          self.finalize_options()
        File "/home/isme/.local/pipx/shared/lib/python3.8/site-packages/setuptools/dist.py", line 900, in finalize_options
          for ep in sorted(loaded, key=by_order):
        File "/home/isme/.local/pipx/shared/lib/python3.8/site-packages/setuptools/dist.py", line 899, in <lambda>
          loaded = map(lambda e: e.load(), filtered)
        File "/home/isme/.local/pipx/shared/lib/python3.8/site-packages/setuptools/_vendor/importlib_metadata/__init__.py", line 196, in load
          return functools.reduce(getattr, attrs, module)
      AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Now I make a temporary virtualenv and install pytrie without problems.

$ pew mktmpenv
created virtual environment CPython3.8.10.final.0-64 in 196ms
  creator CPython3Posix(dest=/home/isme/.local/share/virtualenvs/67476a423031c2a, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/isme/.local/share/virtualenv)
    added seed packages: pip==22.0.4, setuptools==62.1.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
This is a temporary environment. It will be deleted when you exit
Launching subshell in virtual environment. Type 'exit' or 'Ctrl+D' to return.
$ pip install pygtrie
Collecting pygtrie
  Using cached pygtrie-2.4.2.tar.gz (35 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pygtrie
  Building wheel for pygtrie (setup.py) ... done
  Created wheel for pygtrie: filename=pygtrie-2.4.2-py3-none-any.whl size=19061 sha256=15dd3c8247b5d60e51aed00c3ee72ab75391e4c827cf87e393b5cf891eb5f655
  Stored in directory: /home/isme/.cache/pip/wheels/31/03/7b/f685b394a937bc97d2d40908d45aa31f3d9473bca6e9019153
Successfully built pygtrie
Installing collected packages: pygtrie
Successfully installed pygtrie-2.4.2
WARNING: You are using pip version 22.0.4; however, version 22.1 is available.
You should consider upgrading via the '/home/isme/.local/share/virtualenvs/67476a423031c2a/bin/python -m pip install --upgrade pip' command.
$ exit

Now DVC fails to install again, but this time on the configobj package.

$ pipx install dvc[s3]
Fatal error from pip prevented installation. Full pip output in file:
    /home/isme/.local/pipx/logs/cmd_2022-05-18_15.40.25_pip_errors.log

pip seemed to fail to build package:
    configobj>=5.0.6

Some possibly relevant errors from pip install:
    error: subprocess-exited-with-error
    AttributeError: type object 'Distribution' has no attribute '_finalize_feature_opts'
    error: metadata-generation-failed

Error installing dvc from spec 'dvc[s3]'.

Environment information

Output of dvc doctor: Not applicable here.

$ python3 --version
Python 3.8.10
$ pip3 --version
pip 22.1 from /home/isme/.local/lib/python3.8/site-packages/pip (python 3.8)
$ pipx --version
1.0.0
$ lsb_release -a
LSB Version:	core-11.1.0ubuntu2-noarch:printing-11.1.0ubuntu2-noarch:security-11.1.0ubuntu2-noarch
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.4 LTS
Release:	20.04
Codename:	focal
$ uname -r
5.13.0-41-generic

[iainelder]

I solved my installation problem by installing from the debian package repo as pointed out by @daavoo in #7767 (comment) . Thanks!