Re: [matplotlib-devel] Patch suggestion

On Thu, 2005-04-14 at 16:00 -0500, John Hunter wrote: 
> Thanks for the suggestions and patch.  I incorporated frombuffer and
> have been testing it.  I've been testing the performance of frombuffer
> vs fromarray, and have seen some 2-3x speedups but nothing like the
> numbers you are reporting.  [Also, I don't see any detectable memory
> leaks so I don't think you have any worries there]

That kind of speed up is probably more realistic - I probably made a
greater number of optimisations to the python buffer code than to the
numerix code (which I only remembered after posting my first message).
Performance do gains seem greater where memory is limited though as the
reduced memory consumption means less swapping, in cases where the
reduced memory consumption avoids swapping altogether they are, of
course, huge.

> Here is the test script I am using - does this look like a fair test?

It seems to be a fair test - I'd have created the string buffer outside
of the timing (as in reality you wouldn't go through that step) but as
it should be a fairly quick conversion it shouldn't matter too much.

> Another suggestion for Nicholas -- perhaps you want to support MxN,
> MxNx3 and MxNx4 arrays in your frombuffer function?

I could do - the main reason I don't particularly want to is that a
compiler should be able to more easily optimise a simple for(i,i<j,i++)
loop than a series of nested loops and other instructions.  As this code
is only really of use where performance is particularly important I'd
rather keep performance as high as possible - it's easy to generate
buffers in whatever format is necessary.

My suggestion for improved performance would be to allow the Image class
to work directly on the buffer passed to the generator function - I've
started implementing this in c++.  Going with this approach should
improve speed further and again conserve memory for very large images -
and (without making other changes to the Image class) would require that
rgba rather than rgb or intensity was used as an input.

> And a final question -- how are you getting your function into the
> matplotlib image pipeline.  Did you alter the image.py
> AxesImage.set_data function to test whether A is a buffer object?  If
> so, you might want to post these changes to the codebase as well.

I made some alterations to these functions - but they are currently
limited to my own situation.  I will make them general once I've played
around with writable buffers a bit - this will be fairly easily but I
don't want to spend time writing wrapper code until I'm happy with what
I'm wrapping.

Nicholas