Thread: [matplotlib-devel] Qt4 backend: critical bug with PyQt4 v4.6+

Hi,

Some Spyder users have reported a critical bug occuring with matplotlib 
0.99's Qt4 backend and PyQt4 v4.6 (e.g. in Ubuntu Karmic).


Here is the traceback after calling 'plot([])', closing figure and 
calling again 'plot([])' (e.g. in an IPython session with options 
--pylab and --q4thread):

Traceback (most recent call last):
  File "/home/rick/Temp/untitled0.py", line 9, in <module>
    show()
  File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py",
line 63, in show
    manager.window.show()
RuntimeError: underlying C/C++ object has been deleted


I found out that the 'destroyed()' signal (connected in class FigureManagerQT) is never emitted when figure is closed.
As a consequence, SIP is not very happy when trying to draw a deleted object...

I made the following changes to make it work:

# New class to clarify code in FigureManagerQT
class FigureWindow(QtGui.QMainWindow):
    def __init__(self, num, canvas, close_callback):
        super(FigureWindow, self).__init__()
        self.close_callback = close_callback
        self.setAttribute(QtCore.Qt.WA_DeleteOnClose)
        self.setWindowTitle("Figure %d" % num)
        image = os.path.join(matplotlib.rcParams['datapath'],
                             'images', 'matplotlib.png')
        self.setWindowIcon(QtGui.QIcon(image))
        self._destroying = False
        self.setCentralWidget(canvas)
        if matplotlib.is_interactive():
            self.show()
        
    def closeEvent(self, event):
        super(FigureWindow, self).closeEvent(event)
        self.close_callback()

class FigureManagerQT( FigureManagerBase ):
    """
    Public attributes

    canvas      : The FigureCanvas instance
    num         : The Figure number
    toolbar     : The qt.QToolBar
    window      : The qt.QMainWindow
    """

    def __init__( self, canvas, num ):
        if DEBUG: print 'FigureManagerQT.%s' % fn_name()
        FigureManagerBase.__init__( self, canvas, num )
        self.canvas = canvas

        # Give the keyboard focus to the figure instead of the manager
        self.canvas.setFocusPolicy( QtCore.Qt.ClickFocus )
        self.canvas.setFocus()

        self.window = FigureWindow(num, self.canvas, self._widgetclosed)
        self.toolbar = self._get_toolbar(self.canvas, self.window)
        self.window.addToolBar(self.toolbar)
        QtCore.QObject.connect(self.toolbar, QtCore.SIGNAL("message"),
                               self.window.statusBar().showMessage)
# [...]

And we may now remove the "QtCore.QObject.disconnect" for the no longer existing signal 'destroyed()' in method 'FigureManagerQT.
destroy'.

HTH

Cheers,
Pierre

A simpler fix would be:

        class FigureWindow(QtGui.QMainWindow):
            def __init__(self):
                super(FigureWindow, self).__init__()
               
            def closeEvent(self, event):
                super(FigureWindow, self).closeEvent(event)
                self.emit(QtCore.SIGNAL('destroyed()'))

and replacing QtGui.QMainWindow by FigureWindow in FigureManagerQT.

Pierre

Pierre Raybaut a écrit :
> Hi,
>
> Some Spyder users have reported a critical bug occuring with 
> matplotlib 0.99's Qt4 backend and PyQt4 v4.6 (e.g. in Ubuntu Karmic).
>
>
> Here is the traceback after calling 'plot([])', closing figure and 
> calling again 'plot([])' (e.g. in an IPython session with options 
> --pylab and --q4thread):
>
> Traceback (most recent call last):
>  File "/home/rick/Temp/untitled0.py", line 9, in <module>
>    show()
>  File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py",
> line 63, in show
>    manager.window.show()
> RuntimeError: underlying C/C++ object has been deleted
>
>
> I found out that the 'destroyed()' signal (connected in class 
> FigureManagerQT) is never emitted when figure is closed.
> As a consequence, SIP is not very happy when trying to draw a deleted 
> object...
>
> I made the following changes to make it work:
>
> # New class to clarify code in FigureManagerQT
> class FigureWindow(QtGui.QMainWindow):
>    def __init__(self, num, canvas, close_callback):
>        super(FigureWindow, self).__init__()
>        self.close_callback = close_callback
>        self.setAttribute(QtCore.Qt.WA_DeleteOnClose)
>        self.setWindowTitle("Figure %d" % num)
>        image = os.path.join(matplotlib.rcParams['datapath'],
>                             'images', 'matplotlib.png')
>        self.setWindowIcon(QtGui.QIcon(image))
>        self._destroying = False
>        self.setCentralWidget(canvas)
>        if matplotlib.is_interactive():
>            self.show()
>           def closeEvent(self, event):
>        super(FigureWindow, self).closeEvent(event)
>        self.close_callback()
>
> class FigureManagerQT( FigureManagerBase ):
>    """
>    Public attributes
>
>    canvas      : The FigureCanvas instance
>    num         : The Figure number
>    toolbar     : The qt.QToolBar
>    window      : The qt.QMainWindow
>    """
>
>    def __init__( self, canvas, num ):
>        if DEBUG: print 'FigureManagerQT.%s' % fn_name()
>        FigureManagerBase.__init__( self, canvas, num )
>        self.canvas = canvas
>
>        # Give the keyboard focus to the figure instead of the manager
>        self.canvas.setFocusPolicy( QtCore.Qt.ClickFocus )
>        self.canvas.setFocus()
>
>        self.window = FigureWindow(num, self.canvas, self._widgetclosed)
>        self.toolbar = self._get_toolbar(self.canvas, self.window)
>        self.window.addToolBar(self.toolbar)
>        QtCore.QObject.connect(self.toolbar, QtCore.SIGNAL("message"),
>                               self.window.statusBar().showMessage)
> # [...]
>
> And we may now remove the "QtCore.QObject.disconnect" for the no 
> longer existing signal 'destroyed()' in method 'FigureManagerQT.
> destroy'.
>
> HTH
>
> Cheers,
> Pierre
>
>

Hi Pierre,

Thanks for the report. I'll have a look as soon as I get a chance.

Darren

On Wed, Nov 4, 2009 at 5:29 PM, Pierre Raybaut <co...@py...> wrote:
> A simpler fix would be:
>
>        class FigureWindow(QtGui.QMainWindow):
>            def __init__(self):
>                super(FigureWindow, self).__init__()
>
>            def closeEvent(self, event):
>                super(FigureWindow, self).closeEvent(event)
>                self.emit(QtCore.SIGNAL('destroyed()'))
>
> and replacing QtGui.QMainWindow by FigureWindow in FigureManagerQT.
>
> Pierre
>
> Pierre Raybaut a écrit :
>> Hi,
>>
>> Some Spyder users have reported a critical bug occuring with
>> matplotlib 0.99's Qt4 backend and PyQt4 v4.6 (e.g. in Ubuntu Karmic).
>>
>>
>> Here is the traceback after calling 'plot([])', closing figure and
>> calling again 'plot([])' (e.g. in an IPython session with options
>> --pylab and --q4thread):
>>
>> Traceback (most recent call last):
>>  File "/home/rick/Temp/untitled0.py", line 9, in <module>
>>    show()
>>  File "/usr/lib/pymodules/python2.6/matplotlib/backends/backend_qt4.py",
>> line 63, in show
>>    manager.window.show()
>> RuntimeError: underlying C/C++ object has been deleted
>>
>>
>> I found out that the 'destroyed()' signal (connected in class
>> FigureManagerQT) is never emitted when figure is closed.
>> As a consequence, SIP is not very happy when trying to draw a deleted
>> object...
>>
>> I made the following changes to make it work:
>>
>> # New class to clarify code in FigureManagerQT
>> class FigureWindow(QtGui.QMainWindow):
>>    def __init__(self, num, canvas, close_callback):
>>        super(FigureWindow, self).__init__()
>>        self.close_callback = close_callback
>>        self.setAttribute(QtCore.Qt.WA_DeleteOnClose)
>>        self.setWindowTitle("Figure %d" % num)
>>        image = os.path.join(matplotlib.rcParams['datapath'],
>>                             'images', 'matplotlib.png')
>>        self.setWindowIcon(QtGui.QIcon(image))
>>        self._destroying = False
>>        self.setCentralWidget(canvas)
>>        if matplotlib.is_interactive():
>>            self.show()
>>           def closeEvent(self, event):
>>        super(FigureWindow, self).closeEvent(event)
>>        self.close_callback()
>>
>> class FigureManagerQT( FigureManagerBase ):
>>    """
>>    Public attributes
>>
>>    canvas      : The FigureCanvas instance
>>    num         : The Figure number
>>    toolbar     : The qt.QToolBar
>>    window      : The qt.QMainWindow
>>    """
>>
>>    def __init__( self, canvas, num ):
>>        if DEBUG: print 'FigureManagerQT.%s' % fn_name()
>>        FigureManagerBase.__init__( self, canvas, num )
>>        self.canvas = canvas
>>
>>        # Give the keyboard focus to the figure instead of the manager
>>        self.canvas.setFocusPolicy( QtCore.Qt.ClickFocus )
>>        self.canvas.setFocus()
>>
>>        self.window = FigureWindow(num, self.canvas, self._widgetclosed)
>>        self.toolbar = self._get_toolbar(self.canvas, self.window)
>>        self.window.addToolBar(self.toolbar)
>>        QtCore.QObject.connect(self.toolbar, QtCore.SIGNAL("message"),
>>                               self.window.statusBar().showMessage)
>> # [...]
>>
>> And we may now remove the "QtCore.QObject.disconnect" for the no
>> longer existing signal 'destroyed()' in method 'FigureManagerQT.
>> destroy'.
>>
>> HTH
>>
>> Cheers,
>> Pierre
>>
>>
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Matplotlib-devel mailing list
> Mat...@li...
> https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
>

On Wed, Nov 4, 2009 at 5:29 PM, Pierre Raybaut <co...@py...> wrote:
> A simpler fix would be:
>
>        class FigureWindow(QtGui.QMainWindow):
>            def __init__(self):
>                super(FigureWindow, self).__init__()
>
>            def closeEvent(self, event):
>                super(FigureWindow, self).closeEvent(event)
>                self.emit(QtCore.SIGNAL('destroyed()'))
>
> and replacing QtGui.QMainWindow by FigureWindow in FigureManagerQT.

I am pretty sure this is not caused by matplotlib, but rather a
regression in PyQt4 that will be fixed in the next release. There is a
post on the PyQt mailing list titled "Regressions on destruction of
objects?" and a note in the development snapshots changelog. If I try
your example with PyQt4-4.5.4, it works fine, but it crashes with
4.6.1. I will revisit the issue once PyQt4-4.7 is released.

Darren

>
> On Wed, Nov 4, 2009 at 5:29 PM, Pierre Raybaut <co...@py...> wrote:
>   
>> > A simpler fix would be:
>> >
>> > ? ? ? ?class FigureWindow(QtGui.QMainWindow):
>> > ? ? ? ? ? ?def __init__(self):
>> > ? ? ? ? ? ? ? ?super(FigureWindow, self).__init__()
>> >
>> > ? ? ? ? ? ?def closeEvent(self, event):
>> > ? ? ? ? ? ? ? ?super(FigureWindow, self).closeEvent(event)
>> > ? ? ? ? ? ? ? ?self.emit(QtCore.SIGNAL('destroyed()'))
>> >
>> > and replacing QtGui.QMainWindow by FigureWindow in FigureManagerQT.
>>     
>
> I am pretty sure this is not caused by matplotlib, but rather a
> regression in PyQt4 that will be fixed in the next release. There is a
> post on the PyQt mailing list titled "Regressions on destruction of
> objects?" and a note in the development snapshots changelog. If I try
> your example with PyQt4-4.5.4, it works fine, but it crashes with
> 4.6.1. I will revisit the issue once PyQt4-4.7 is released.
>
> Darren
>   
I completely agree -- in the meantime I checked that despite the 
QMainWindow instance has been destroyed, the 'destroyed()' signal is not 
emitted, so the problem is definitely coming from PyQt4, not matplotlib.

However, to help unfortunate users of PyQt4 v4.6 and v4.6.1, I've added 
the following workaround in Spyder (Spyder is monkey patching matplotlib 
anyway, to integrate mpl dockable figures in Spyder's GUI) -- other 
users may be interested to patch their matplotlib installation 
(matplotlib/backends/backend_qt4.py):

# Add this before "FigureManagerQT" class
class FigureWindow(QMainWindow):
    def __init__(self):
        super(FigureWindow, self).__init__()      
    def closeEvent(self, event):
        super(FigureWindow, self).closeEvent(event)
        if PYQT_VERSION_STR.startswith('4.6'):
            self.emit(SIGNAL('destroyed()'))
# Replace "QtGui.QMainWindow" by "FigureWindow" in "FigureManagerQT"'s 
constructor

I hope this will be fixed in v4.6.2.

Cheers,
Pierre

On Sat, Nov 7, 2009 at 9:53 AM, Pierre Raybaut <co...@py...> wrote:
>>
>> On Wed, Nov 4, 2009 at 5:29 PM, Pierre Raybaut <co...@py...>
>> wrote:
>>
>>>
>>> > A simpler fix would be:
>>> >
>>> > ? ? ? ?class FigureWindow(QtGui.QMainWindow):
>>> > ? ? ? ? ? ?def __init__(self):
>>> > ? ? ? ? ? ? ? ?super(FigureWindow, self).__init__()
>>> >
>>> > ? ? ? ? ? ?def closeEvent(self, event):
>>> > ? ? ? ? ? ? ? ?super(FigureWindow, self).closeEvent(event)
>>> > ? ? ? ? ? ? ? ?self.emit(QtCore.SIGNAL('destroyed()'))
>>> >
>>> > and replacing QtGui.QMainWindow by FigureWindow in FigureManagerQT.
>>>
>>
>> I am pretty sure this is not caused by matplotlib, but rather a
>> regression in PyQt4 that will be fixed in the next release. There is a
>> post on the PyQt mailing list titled "Regressions on destruction of
>> objects?" and a note in the development snapshots changelog. If I try
>> your example with PyQt4-4.5.4, it works fine, but it crashes with
>> 4.6.1. I will revisit the issue once PyQt4-4.7 is released.
>>
>> Darren
>>
>
> I completely agree -- in the meantime I checked that despite the QMainWindow
> instance has been destroyed, the 'destroyed()' signal is not emitted, so the
> problem is definitely coming from PyQt4, not matplotlib.
>
> However, to help unfortunate users of PyQt4 v4.6 and v4.6.1, I've added the
> following workaround in Spyder (Spyder is monkey patching matplotlib anyway,
> to integrate mpl dockable figures in Spyder's GUI) -- other users may be
> interested to patch their matplotlib installation
> (matplotlib/backends/backend_qt4.py):
>
> # Add this before "FigureManagerQT" class
> class FigureWindow(QMainWindow):
>   def __init__(self):
>       super(FigureWindow, self).__init__()         def closeEvent(self,
> event):
>       super(FigureWindow, self).closeEvent(event)
>       if PYQT_VERSION_STR.startswith('4.6'):
>           self.emit(SIGNAL('destroyed()'))
> # Replace "QtGui.QMainWindow" by "FigureWindow" in "FigureManagerQT"'s
> constructor
>
> I hope this will be fixed in v4.6.2.

Me too. And thank you for posting the report and a workaround.

Howdy,

On Sat, Nov 7, 2009 at 12:30 PM, Darren Dale <dsd...@gm...> wrote:
> Me too. And thank you for posting the report and a workaround.

Quick question: would it be worth adding this monkeypatch to mpl
proper?  Right now, the qt4 backend is effectively unusable out of the
box in distros like Karmic.  Which is a bummer, because with the
ipython sitting on my laptop, one can now load 'pylab' at any time
during a session:

maqroll[scratch]> ip
Python 2.6.4 (r264:75706, Dec  7 2009, 18:45:15)
Type "copyright", "credits" or "license" for more information.

IPython 0.11.bzr.r1219 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import sys

In [2]: 'matplotlib' in sys.modules
Out[2]: False

In [3]: %pylab wx
Activating matplotlib with backend: WXAgg

  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.

Switching IPython gui support to: wx True

In [4]: 'matplotlib' in sys.modules
Out[4]: True

In [5]: plot(sin(linspace(0,2*pi,200)))
Out[5]: [<matplotlib.lines.Line2D object at 0xae0dccc>]


This is starting to look very promising, but unfortunately:

- right now we don't have gui switching support for Qt3 at all in
ipython.  Help is welcome, but I have no clue if it's easy/hard or
even needed much anymore...

- qt4 is unusable with the system's qt/pyqt...

So perhaps a local patch would be worth it, no?  I can confirm that
with the attached patch, the new ipython support works:

In [1]: %pylab qt
Activating matplotlib with backend: Qt4Agg

  Welcome to pylab, a matplotlib-based Python environment.
  For more information, type 'help(pylab)'.


In [2]: run simpleplot.py

In [3]: close('all')

In [4]: run simpleplot.py


whereas before, I'd get the same nasty error mentioned above.

The patch now has no run-time impact (I modified Pierre's code a bit
so the check is done only once), but I'm not about to commit something
in the Qt backend without someone else's eyes, especially Darren's :)

Cheers,

f

On Wed, Dec 30, 2009 at 11:11 PM, Fernando Perez <fpe...@gm...> wrote:
> Howdy,
>
> On Sat, Nov 7, 2009 at 12:30 PM, Darren Dale <dsd...@gm...> wrote:
>> Me too. And thank you for posting the report and a workaround.
>
> Quick question: would it be worth adding this monkeypatch to mpl
> proper?  Right now, the qt4 backend is effectively unusable out of the
> box in distros like Karmic.

I have been resistant to committing this patch because (in my opinion)
mpl should not have to provide workarounds for bugs in package X on OS
Y, distribution Z. I think this particular issue was fixed when
PyQt4-4.6.2 was released. But its time to get practical, I suppose.
The patch looks fine, I just checked it into the trunk.

Darren

On Thu, Dec 31, 2009 at 4:54 AM, Darren Dale <dsd...@gm...> wrote:
> I have been resistant to committing this patch because (in my opinion)
> mpl should not have to provide workarounds for bugs in package X on OS
> Y, distribution Z. I think this particular issue was fixed when
> PyQt4-4.6.2 was released. But its time to get practical, I suppose.
> The patch looks fine, I just checked it into the trunk.

Thanks!  As the zen goes, practicality beats purity :)  I understand
your reluctance though, it's annoying to pepper mpl's code with this
kind of junk.

Happy New Year!

f

2009/12/31 Fernando Perez <fpe...@gm...>:
> On Thu, Dec 31, 2009 at 4:54 AM, Darren Dale <dsd...@gm...> wrote:
>> I have been resistant to committing this patch because (in my opinion)
>> mpl should not have to provide workarounds for bugs in package X on OS
>> Y, distribution Z. I think this particular issue was fixed when
>> PyQt4-4.6.2 was released. But its time to get practical, I suppose.
>> The patch looks fine, I just checked it into the trunk.
>
> Thanks!  As the zen goes, practicality beats purity :)  I understand
> your reluctance though, it's annoying to pepper mpl's code with this
> kind of junk.
>
> Happy New Year!
>
> f
>

I completely agree. If developers were all doing their "job" in time,
this should not be necessary and Darren's position would be perfectly
right and justified. However, especially on certain open-source
libraries, things are not moving as fast as they should. For example,
in Spyder I had to re-implement a console-oriented text editor widget
with Scintilla-like features because QScintilla's widget had a
performance issue with very long lines (which is almost undectectable
when using it as a simple text editor but it may slow down display
when using it as a console widget). This bug was fixed just a few days
after being reported but there has been no release since then (August
2009). So, to make it work, I had to do this big workaround until the
next release of QScintilla has spread on every platform (i.e. not
until a year I guess). In terms of code refactoring (or purity...),
this was not very satisfying but now Spyder works perfectly because I
stopped saying "it's not my fault, it's QScintilla's".

So even if I agree with Darren on the fact that libraries such as
matplotlib should not provide this kind of workaround, I also think
that -at some point- one has to get practical indeed!

Happy new year guys!

Cheers,
Pierre