Support zarr v3

[Koncopd]

Please make sure these conditions are met

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of anndata.
• (optional) I have confirmed this bug exists on the master branch of anndata.

Report

See

Versions

latest

[mbstadler]

We also noticed this issue (our package depends on anndata as well as zarr).
This guide was useful to update our own code to zarr v3: https://zarr.readthedocs.io/en/latest/user-guide/v3_migration.html#

[flying-sheep]

Huh, I thought we had a feature request for this already! OK, I’ll make this one into it.

Maybe we should have added a zarr extra that you can use to say “get the newest supported zarr version”, instead of just being able to see that we don’t support it yet from our test pin:

anndata/pyproject.toml

Line 91 in 21d191d

"zarr<3.0.0a0",

See

• (fix): upper bound zarr at runtime (and in docs + benchmark) #1819
• (feat): support for zarr-python>=3  #1726

[jimmymathews]

This is not exactly the same thing but I noticed that pyproject.toml does not declare zarr<3 in dependencies (in fact, zarr is not in the primary dependencies at all), even though this constraint is enforced at runtime here if zarr happens to be installed:

anndata/src/anndata/compat/__init__.py

Line 96 in 9838253

if Version(zarr.__version__).major > 2:

So one can get anndata-triggered errors as follows:

# This is fine:
pip install anndata
python -c 'from anndata import AnnData'

# This raises exception:
pip install anndata
pip install zarr
python -c 'from anndata import AnnData'

The error is:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    from anndata import AnnData
  File "/Users/user/venv/lib/python3.13/site-packages/anndata/__init__.py", line 29, in <module>
    from ._core.anndata import AnnData
  File "/Users/user/venv/lib/python3.13/site-packages/anndata/_core/anndata.py", line 25, in <module>
    from .. import utils
  File "/Users/user/venv/lib/python3.13/site-packages/anndata/utils.py", line 15, in <module>
    from ._core.sparse_dataset import BaseCompressedSparseDataset
  File "/Users/user/venv/lib/python3.13/site-packages/anndata/_core/sparse_dataset.py", line 32, in <module>
    from .._settings import settings
  File "/Users/user/venv/lib/python3.13/site-packages/anndata/_settings.py", line 17, in <module>
    from anndata.compat import CAN_USE_SPARSE_ARRAY
  File "/Users/user/venv/lib/python3.13/site-packages/anndata/compat/__init__.py", line 97, in <module>
    raise ImportError("zarr-python major version > 2 is not supported")
ImportError: zarr-python major version > 2 is not supported

Takeaway: Until zarr>2 is supported, probably zarr<3 should be declared in pyproject or else this runtime check in compat.py should be removed.

[ilan-gold]

@jimmymathews Yes, that's right but I wanted to stay on the safe side and not require zarr until 0.12. We will require it going forward.