[Python][CI] Substrait nightly job is failing due to wrong invocation of substrait consumer-testing

[raulcd]

Describe the bug, including details regarding any error messages, version, and platform.

The integration test for substrait (test-conda-python-3.10-substrait) has been failing for the last 21 days with:

 Substrait Integration Tests
Validating imports
Executing pytest
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --producer isthmus --consumer acero
  inifile: /consumer-testing/pytest.ini
  rootdir: /consumer-testing

The execution fails when running:

pytest -r s substrait_consumer/tests/functional/extension_functions/test_boolean_functions.py --producer isthmus --consumer acero

Component(s)

Continuous Integration, Python

[raulcd]

This is not the first time the job fails.

• [Python][CI] Substrait nightly job is failing due to wrong invocation of substrait consumer-testing #44726
• [CI][Python] test-conda-python-3.10-substrait build failing  #43045
• [CI] Test substrait integration job has been failing for some time #39425

Should we keep maintaining it?

[raulcd]

cc @westonpace

[raulcd]

@jorisvandenbossche @pitrou @kou this job has been failing for 5 weeks and it does not seem to be maintained. Should we propose to remove it? I don't think we ever detected a failure on the Arrow side on this job but always have found it failing due to changes around substrait-consumer. I've opened an issue on the substrait repo and suggested the removal.

[kou]

+1 for removal.

Another idea: How about bundling a subset of consumer-testing and updating it on our own time?

It seems that we only uses test_boolean_functions.py:

arrow/ci/scripts/integration_substrait.sh

Line 31 in ef00568

pytest -r s substrait_consumer/tests/functional/extension_functions/test_boolean_functions.py --producer isthmus --consumer acero

[raulcd]

I am going to remove the job. It seems those test files were removed from upstream and I am unsure how we are supposed to be testing at the moment:
substrait-io/consumer-testing@b9a2c2d#diff-3b4dab90628ffb3cc9cc2d1340fac6a507e7ed9a2a0e4fa2b6e81b9be54e4547
I tried to navigate the related PRs but it's unclear to me.