How to configure?

[ilyakamens]

Hi! I'm having trouble getting this to work. I've installed the plugin via our requirements.txt file. We use docker, and so I set the env var as mentioned. I see the following in the test output:

::error file=foo/tests/unit/test_bar.py,line=13::TestBar.test_fail%0A%0Aassert False

That path looks correct to me. foo is a top-level directory in our repository. Is there something else I need to do to get this to work? We use a Jenkins pipeline + shell command to run tests. Thanks!

[daveisfera]

I'm having the same issue. I'm running inside of a container, I've set GITHUB_ACTIONS=true and verified that it's set when running the test, but the output is identical.

[edgarrmondragon]

inside of a container

@daveisfera or @ilyakamens Can you share an example of how you're doing that in GitHub Actions or set up a MRE repo?

(My suspicion is that the container's output is being redirected to stdout and GitHub doesn't detect it there?)

[daveisfera]

I mean something even simpler than that. The actual output isn't changed at all when this is installed and that's with both GITHUB_ACTIONS=true set and not. I assume that this is supposed to change the output so that GitHub Actions parses it to tie it to the code or is that not correct?

[edgarrmondragon]

The actual output isn't changed at all when this is installed and that's with both GITHUB_ACTIONS=true set and not. I assume that this is supposed to change the output so that GitHub Actions parses it to tie it to the code or is that not correct?

Yeah, consider this working example:

FROM python:3.13-slim

RUN pip install pytest pytest-github-actions-annotate-failures

RUN echo "def test_fail():\n    assert False" > test_fail.py

ENV GITHUB_ACTIONS=true
CMD ["pytest", "test_fail.py"]

Then

1. build the image

   docker build -t pytest-gh 

2. run it

   $ docker run --rm pytest-gh            
   ============================= test session starts ==============================
   platform linux -- Python 3.13.1, pytest-8.3.4, pluggy-1.5.0
   rootdir: /
   plugins: github-actions-annotate-failures-0.3.0
   collected 1 item
   
   test_fail.py F::error file=test_fail.py,line=2::test_fail%0A%0Aassert False
                                                           [100%]
   
   =================================== FAILURES ===================================
   __________________________________ test_fail ___________________________________
   
      def test_fail():
   >       assert False
   E       assert False
   
   test_fail.py:2: AssertionError
   =========================== short test summary info ============================
   FAILED test_fail.py::test_fail - assert False
   ============================== 1 failed in 0.02s ===============================

   so you can see the output is there and in the right format

3. confirm it's sent to stderr

   $ docker run --rm pytest-gh > /dev/null
   ::error file=test_fail.py,line=2::test_fail%0A%0Aassert False

[daveisfera]

I just realized that I was expecting the output to be completely different. I thought it would change the actual output in some way and not that it output a new type of line to stderr, so that was just a silly assumption on my part and I can now see that it's working

[daveisfera]

Ok, but I did notice that the link in the error it generates doesn't open the line of the failed test and appears to just open the commit that failed. Is that an issue with the output that it makes or with how GitHub Actions handles it?

[edgarrmondragon]

Ok, but I did notice that the link in the error it generates doesn't open the line of the failed test and appears to just open the commit that failed. Is that an issue with the output that it makes or with how GitHub Actions handles it?

@daveisfera do you have an example of that?

[daveisfera]

Here's an example:
https://github.com/flairNLP/flair/actions/runs/13891740403