On 3/12/07, D. Stimits wrote:

I have not found any python-fu way to "close" a file, or to reclaim memory after creating an image or layer. Is there such a thing? Is there instead some sort of garbage collection?

hahahaha!

gimp.delete(image)

or gimp.delete(layer) #only do this if the layer isn't attached to an image. If it is attached to an image, then deleting the image will delete all the layers and other things -- no need to delete them directly.

This *is* in the pygimp documentation (plug-ins/pygimp/doc/pygimp.html in the source tree)! RTFM!

On Sunday 11 March 2007 20:54, D. Stimits wrote:

I have not found any python-fu way to "close" a file, or to reclaim memory after creating an image or layer. Is there such a thing? Is there instead some sort of garbage collection?

Speculating about why it claimed it was out of memory, I'm wondering if the hard drive simply was too slow responding. It's a laptop, running with core duo, entirely in ram. The drive is thus much slower than the ability to consume ram. How does gimp determine that there is no space left on the device...is it a timeout in addition to other means?

The normal python way would be to set the object for destruction after it is no longer used.

However, it may well be possible that the GIMP bindings to python are not deleting GIMP images creted from within python, as it should be done.

I had, myself, never thought of using a script o create so many images.

As you rerro r mesage is "no space left on device" - please try to cehck gimp preferences for tmp dirs, maybe it is using another partition that might be getting filled.

The increaseof RAM usage for itself however shows there is something wrong going on - could you please fill a bug report about this at bugzilla.gnome.org ?

If possible, please attach a minimalistic version of your script taht would trigger the problem.

Thank you,

js ->

I am hereforth asking you to apologise, if possible, both in private

and on the list for this message. We are runing a project needing

I cannot sincerely do so, in this instance. I do not apologize for expressing my amusement, unless I judge that the situation was genuinely worsened by my behaviour.

It's certainly true that my message can be misinterpreted -- on the net, it's probably wisest to just assume no hidden meanings -- if it looks like a fish, assume it is a fish (and nothing more).

much of new developers and contributors, and are in no position of

theating anyone like this. Please perceive I am no "police" - I am asking this personally, but I also have promised the people organizing the LGM I'd tlak to gimp developers to try to make the enviroment more frindly to newcomers.

In my observation, friendly sometimes is not polite -- for instance, answering as I did rather than giving a detached commentary free of any emotional context at all.
This is why I view the way I replied as better (and certainly more friendly) than most previous mails of mine on this list.

(btw, having to explicitely delete a python object __is__ a bug -

objects that are seeing in python should be garbage collected when they are no longer referecenced. )

I avoided commenting on that because it wasn't clear whether they actually were no longer referenced. I certainly agree that there is something wrong with the gimp.delete convention here -- it only behaves how I'd expect sometimes.

I'd definitely like to see more PyGimp developers myself -- hardly anyone seems to make as extensive use of it as I do. Perhaps a standard module path for pygimp plugins could be worked out (arguably, the PDB removes this need, but the PDB is cranky to use compared to Python modules)

On Mon, 12 Mar 2007 00:54:56 +0100, D. Stimits wrote:

Speculating about why it claimed it was out of memory, I'm wondering if the hard drive simply was too slow responding. It's a laptop, running with core duo, entirely in ram. The drive is thus much slower than the ability to consume ram. How does gimp determine that there is no space left on the device...is it a timeout in addition to other means?

I saw something a little like this recently. I cant document it thoroughly but here's a resumé in case it's related.

I tried to open about eight multi-megabyte jpeg images in one open operation not wanting to go through the labourious open dialogue eight times and not noticing quite how big they were. I was half expecting some error after a few were opened but expected error trapping to handle it correctly.

What I ended up with was a completely unstable mess which I had to clean up with xkill , restart the xserver and a bit of ps and kill to clean up the residue.

Now this could be regarded as extreme conditions and represents a good stress test. It seems running out of RAM and/or swap is not properly error trapped.

I just throw this info in in case it relates the this script issue posted.

gg

Von: "David Gowers"

I am hereforth asking you to apologise, if possible, both in private

and on the list for this message. We are runing a project needing

I cannot sincerely do so, in this instance. I do not apologize for expressing my amusement, unless I judge that the situation was genuinely worsened by my behaviour.

We could make it the policy that anyone who does shout "RTFM!" has to:

- make sure that there is a FM - make sure that it is up-to-date
- provide the URL

In this case, this would be like:

"RTFM! http://www.gimp.org/docs/python/index.html"

I'll leave the check whether this is up-to-date for David as an exercise :)

HTH, Michael

I think you have hit an actual bug, and that it doesn't have anything to do with python, because I have encountered a similar thing in a modified version of gimpressionist that I worked up, written purely in C. I believe that there is some sort of memory leak that causes gimp in some situations to maintain reference to tiles that are no longer being used in any way, and it is something that shows up when you create and delete layers over and over again, in a certain way. So it would probably be useful for you to file a bug report about this, if you would.

The memory that is being exhausted, by the way, is the "swap" area that gimp allocates on the hard disk each time you run it. The tile manager moves tile data there if space is needed.

-- Bill

______________ ______________ ______________ ______________ Sent via the CNPRC Email system at primate.ucdavis.edu

Hi,

On Mon, 2007-03-12 at 06:51 -0800, William Skaggs wrote:

I think you have hit an actual bug, and that it doesn't have anything to do with python, because I have encountered a similar thing in a modified version of gimpressionist that I worked up, written purely in C. I believe that there is some sort of memory leak that causes gimp in some situations to maintain reference to tiles that are no longer being used in any way, and it is something that shows up when you create and delete layers over and over again, in a certain way. So it would probably be useful for you to file a bug report about this, if you would.

It would only be useful if that bug report included a way to reproduce the problem. And it would probably be helpful to discuss that code example beforehand.

I am not saying that there are no memory leaks in GIMP. But we are frequently testing the software for this and I am not aware of any major memory leaks such as the one that Bill describes here.

The memory that is being exhausted, by the way, is the "swap" area that gimp allocates on the hard disk each time you run it. The tile manager moves tile data there if space is needed.

Actually, it doesn't do that when space is needed, but when the memory allocated for tiles exceeds the limit configured as 'tile-cache-size'.

Sven

Hi,

On Tue, 2007-03-13 at 22:18 -0600, D. Stimits wrote:

There could be a mix of a gimp bug, and definitely some portion of my plug-in. Although I know for a fact that I had a memory leak that was my fault, the reaction to the leak was unexpected. The system in question has only a /boot and /, with well in excess of 30 GB unused (lots of inodes as well). For a moment I thought that perhaps it was a max open file descriptor limit...which it could still be, but the system itself has a default setting of a max of around 76,618. The thing I did not expect was that it had well over a gig of ram left unused, no swap used, 30 gig of unused drive, and it still thought the hard drive was full. On the other hand, it might not be worth pursuing when it requires a memory leak for it to show up.

How is that unexpected behavior? That's exactly what I would expect if you configure the GIMP tile-cache-size to be smaller than the amount of physical memory. GIMP will then start to swap tiles out even before it runs out of memory. And if you don't delete images, you will end up with your harddisk filled up.

Sven

[...] The thing I did not
expect was that it had well over a gig of ram left unused, no swap used, 30 gig of unused drive, and it still thought the hard drive was full. [...]

It might help if I explain a little more about how GIMP handles memory. GIMP does not rely on the operating system for swapping. When you start GIMP, it creates a file, of fixed size which you designate in the Preferences, to serve as a swap area. You also tell GIMP, in the Preferences, how much RAM it is allowed to use. If GIMP needs more memory, it moves some of its data from RAM into the swap file.

What is happening to you is that the swap file is filling up, because data is accumulating there that ought to be cleared away. You could delay the failure by increasing the size of the swap file (called the "tile cache") in Preferences, but really, in the work flow you described, it ought not to be filling up, so there is a bug either in your code or in GIMP -- and as I said earlier, I think there is probably a bug in GIMP.

(I am not 100% certain that I got all the details right here. Presumably Sven or Mitch will correct any mis-statement.)

-- Bill

______________ ______________ ______________ ______________ Sent via the CNPRC Email system at primate.ucdavis.edu

On 3/14/07, William Skaggs wrote:

What is happening to you is that the swap file is filling up, because data is accumulating there that ought to be cleared away.

btw, is it possible to provide some feedback to users that available resources are getting pretty close to zero?

Alexandre

On Wednesday 14 March 2007 17:01, William Skaggs wrote:

It might help if I explain a little more about how GIMP handles memory.  GIMP does not rely on the operating system for swapping. When you start GIMP, it creates a file, of fixed size which you designate in the Preferences, to serve as a swap area.  You also tell GIMP, in the Preferences, how much RAM it is allowed to use. If GIMP needs more memory, it moves some of its data from RAM into the swap file.

Strange... I have 1GB of RAM, and 1GB of swap. My tile cache is set to 512MB, and undo history to 256MB. I recently tried to edit a big pano. After one color correction, The RAM get full, and the system swap too! Then, all stopped working.

According to your explanation, it should not do that. Do I missed something?

Hi,

On Wed, 2007-03-14 at 09:01 -0700, William Skaggs wrote:

It might help if I explain a little more about how GIMP handles memory. GIMP does not rely on the operating system for swapping. When you start GIMP, it creates a file, of fixed size which you designate in the Preferences, to serve as a swap area.

Sorry, but I have to correct you. The size of the swap file is of course not fixed. It will grow and may even shrink with the amount of swap space that is needed. I said that it may shrink because there is currently no code to deal with holes in the swap file. So it will only shrink if the tiles stored at its end are released.

What is happening to you is that the swap file is filling up, because data is accumulating there that ought to be cleared away. You could delay the failure by increasing the size of the swap file (called the "tile cache") in Preferences

The tile-cache-size in the Preferences is the amount of memory GIMP will use before starting to swap tiles out to the swap file. It does not specify the size of the swap file.

Sven