misc IPython 9 compatibility

[minrk]

just needed a couple ANSI constants

avoids losing the behavior dropped by #924

[minrk]

@Carreau you don't need to do the IPython release if you don't want, there are a couple other 9.0 compatibility fixes needed and I can do the release today.

[Carreau]

no that's ok, I was expecting 9.0 to break a couple of things, and was quasi certain there would be a 9.0.1

[minrk]

I also imported the internal core.display.display here, when it should have always been IPython.display. Fixing that, too.

[Carreau]

IPython 9.0.1 should be out.

[minrk]

There also appears to be something different about matplotlib registration, because calling %matplotlib inline is resulting in:

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/IPython/core/interactiveshell.py:3634, in InteractiveShell.enable_matplotlib(self, gui)
   3631     import matplotlib_inline.backend_inline
   3633 from IPython.core import pylabtools as pt
-> 3634 gui, backend = pt.find_gui_and_backend(gui, self.pylab_gui_select)
   3636 if gui != None:
   3637     # If we have our first gui selection, store it
   3638     if self.pylab_gui_select is None:

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/IPython/core/pylabtools.py:349, in find_gui_and_backend(gui, gui_select)
    347 else:
    348     gui = _convert_gui_to_matplotlib(gui)
--> 349     backend, gui = backend_registry.resolve_gui_or_backend(gui)
    351 gui = _convert_gui_from_matplotlib(gui)
    352 return gui, backend

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib/backends/registry.py:409, in BackendRegistry.resolve_gui_or_backend(self, gui_or_backend)
    407     return self.resolve_backend(gui_or_backend)
    408 except Exception:  # KeyError ?
--> 409     raise RuntimeError(
    410         f"'{gui_or_backend} is not a recognised GUI loop or backend name")

RuntimeError: 'inline is not a recognised GUI loop or backend name

which I guess means the inline backend hasn't been registered yet? Weirdly, this same call works fine in both terminal IPython and jupyter-console in the same env, so I don't know what Ipyhon/ipykernel usually do doing startup that ipyparallel is skipping somehow.

[minrk]

Ah, apparently ipykernel sets MPLBACKEND=module://matplotlib_inline.backend_inline which is apparently required for %matplotlib inline to work. Is that intended? I'm guessing not. It seems like the inline backend discovery isn't working without this.

[minrk]

maybe it's something weird about my environment, but calling enable_gui_pylab which by default does nothing but set a default value for MPLBACKEND seems to fix it.

[Carreau]

most of the inline matplotlib stuff should be in matplotlib-inline now.

[minrk]

most of the inline matplotlib stuff should be in matplotlib-inline now.

Yes, but there seem to be some circular things in how the inline backend is loaded. I'm seeing:

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib_inline/__init__.py:1
----> 1 from . import backend_inline, config  # noqa
      2 __version__ = "0.1.7"  # noqa

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib_inline/backend_inline.py:228
    225             ip.events.register('post_run_cell', configure_once)
--> 228 _enable_matplotlib_integration()
    231 def _fetch_figure_metadata(fig):

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib_inline/backend_inline.py:213, in _enable_matplotlib_integration()
    212 ip = get_ipython()
--> 213 backend = get_backend()
    214 if ip and backend in ('inline', 'module://matplotlib_inline.backend_inline'):

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib/__init__.py:1323, in get_backend(auto_select)
   1322 if auto_select:
-> 1323     return rcParams['backend']
   1324 else:

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib/__init__.py:797, in RcParams.__getitem__(self, key)
    796         from matplotlib import pyplot as plt
--> 797         plt.switch_backend(rcsetup._auto_backend_sentinel)
    799 return self._get(key)

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib/pyplot.py:411, in switch_backend(newbackend)
    410 try:
--> 411     switch_backend(candidate)
    412 except ImportError:

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib/pyplot.py:523, in switch_backend(newbackend)
    522 # Make sure the repl display hook is installed in case we become interactive.
--> 523 install_repl_displayhook()

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib/pyplot.py:323, in install_repl_displayhook()
    322 if ipython_gui_name:
--> 323     ip.enable_gui(ipython_gui_name)
File ~/.virtualenvs/ip9/lib/python3.11/site-packages/ipykernel/zmqshell.py:494, in ZMQInteractiveShell.enable_gui(self, gui)
    493 except ValueError as e:
--> 494     raise UsageError("%s" % e) from e

UsageError: Invalid GUI request 'macosx', valid ones are:dict_keys(['inline', 'nbagg', 'webagg', 'notebook', 'ipympl', 'widget', None, 'qt', 'qt5', 'qt6', 'wx', 'tk', 'gtk', 'gtk3', 'osx', 'asyncio'])

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
Cell In[3], line 1
----> 1 backend_registry.resolve_gui_or_backend("inline")

File ~/.virtualenvs/ip9/lib/python3.11/site-packages/matplotlib/backends/registry.py:409, in BackendRegistry.resolve_gui_or_backend(self, gui_or_backend)
    407     return self.resolve_backend(gui_or_backend)
    408 except Exception:  # KeyError ?
--> 409     raise RuntimeError(
    410         f"'{gui_or_backend} is not a recognised GUI loop or backend name")

RuntimeError: 'inline is not a recognised GUI loop or backend name

I'm not sure why it's fully loading the default backend with event loop integration (and then doing something incorrect with the event loop hook) before loading the inline backend.

This seems like possibly a bug in the inline backend, but maybe also a bug in IPython and/or ipykernel, because enable_gui('macosx') should never be called, and enable_gui('osx') shouldn't be called as part of enabling the inline backend. Might be smething amiss in some of the deprecations in pylabtools?

[minrk]

Ah, looks like the call to convert from matplotlib is added to terminal IPython in a place that's not inherited by ipykernel. I guess ipykernel needs an update for that.

But still, I think the hookup of the inline backend should perhaps avoid fully loading the default backend, if possible.

I think maybe that's what this bit in pylabtools.activate_matplotlib aims to do, but at least in IPython 9, it is not running early enough to have the desired effect.

[minrk]

Looks like this issue is already open as ipython/matplotlib-inline#25 and fixed by ipython/matplotlib-inline#38 so a fresh release of matplotlib-inline will solve it.