RSS/Atom feed Twitter
Site is read-only, email is disabled

Some tests of GIMP startup time over NFS

This discussion is connected to the gimp-developer-list.gnome.org mailing list which is provided by the GIMP developers and not related to gimpusers.com.

This is a read-only list on gimpusers.com so this discussion thread is read-only, too.

2 of 2 messages available
Toggle history

Please log in to manage your subscriptions.

Some tests of GIMP startup time over NFS Raphaël Quinet 02 Dec 15:30
  Some tests of GIMP startup time over NFS Sven Neumann 05 Dec 03:57
Raphaël Quinet
2004-12-02 15:30:18 UTC (over 19 years ago)

Some tests of GIMP startup time over NFS

Hi all,

On Monday, there was a brief discussion on irc about the GIMP startup time when the files are fetched over NFS. Using NFS or any other remote file system, all read/writes/stats are more expensive than from a local disk. The discussion was about what could be improved to make the GIMP load faster. I initally thought that some of the delays that are more visible over NFS were due to the way all plug-ins are stat()'ed to check their timestamp, but the test results included below show that I was wrong. So I would like to share these results with those would could be interested (I am unable to re-join irc for the moment). Note that the discussion was about the time that it takes for the GIMP to scan all its plug-ins, brushes, patterns, and other data files (i.e., the time during which the splash screen is shown) and not about the time taken to open an image, create the menus or related stuff that is found in Bugzilla.

I made some tests from a rather old Sun Ultra10 box running Solaris 8, which is connected over 100Mbps Ethernet to an old NFS server containing the GIMP installation and to another slightly faster NFS server hosting my home directory. Using the Solaris "truss -d" command (similar to "strace -tt" for Linux), I timed the startup time of the gimp-2.0, the stable version that I installed on our NFS server. I created a first trace file on Monday (about 2.5 MB of trace messages), which I uploaded to:
http://wilber.gimp.org/~raphael/gimp-2.0-solaris-syscalls.txt That file was good but not perfect because a few files were still in the NFS cache when I did the test, so I re-did a new trace later. Unfortunately I cannot upload it for the moment because I have some problems accessing wilber.gimp.org, but the results are not very different from the previous run.

Here is a summary of the timings (from the new file, unfortunately): - from 0 to 1.76 seconds: loading of the app and all libs (gtk, X11...) - until 3.24 (diff: 1.48): X client init, connection to X server - until 5.23 (diff: 1.99): get pixbuf loader, load splash, init pango, get font, display splash - until 5.43 (diff: 0.20): load environ, parasiterc, init progress bar - until 6.34 (diff: 0.91): load 48 brushes - until 7.05 (diff: 0.71): load 58 patterns - until 8.24 (diff: 1.19): load 39 palettes - until 8.76 (diff: 0.52): load 78 gradients - until 9.55 (diff: 0.79): load fonts, templates, modules, sessionrc... - until 12.07 (diff: 2.52): load tool presets - until 12.24 (diff: 0.17): stat all plug-ins - until 12.69 (diff: 0.45): read pluginrc - until 14.63 (diff: 1.94): update progress bar during init of 156 plug-ins - until 25.75 (diff:11.12): pipe & fork script-fu, init 95 scripts and pdb - until 30.56 (diff: 4.81): read menurc, devicerc, final init, display docks - after that, the GIMP is idle for some seconds and then I close it

I repeated the same test several times, waiting a couple of hours and doing some other tasks between each test in order to flush the NFS caches (client and server). In all cases, the startup time was around 30 seconds (between 30.1 and 30.6) so the test can be reproduced easily and the results are consistent.

Some comments: - It takes about 5 seconds until the splash is displayed, but it would be difficult to improve that time because most of it is spent loading the main executable and libraries over the network. It would be possible to show a splash screen earlier by using some hacks such as loading it from a small statically-linked executable written only for that purpose but this is not worth the effort IMHO (some commercial apps do it that way). - The gih brush loader could be improved because it reads its files one byte at a time, but that has only a minor impact on the overall delay (the whole brush file is cached locally anyway). We could not save more than a fraction of a second by improving this. - Loading palettes is slower than I expected. Not a problem, though. - Loading tool presets is slower than I expected. The time is spent in the GIMP itself and not loading the very small files, so NFS is probably not making much of a difference here. - Doing the stat64() for all plug-ins is faster than I expected, so this is not the main cause of delays as I thought initially. - Initializing script-fu takes a lot of time (more than 10 seconds), mostly due to the large amount of inter-process communication. This delay is not related to NFS. It would probably be similar on a x86 machine running at less than 500MHz. - If most files are already in the local NFS cache before starting the GIMP, then it starts about 7 seconds faster (23 seconds instead of 30 seconds).

Conclusion: the additional delay due to potentially inefficient usage of NFS is not that big, if we keep in mind that it would be difficult to reduce the initial loading time for the main app and libs. The other potential optimizations are not specific to NFS.

-Raphaël

P.S.: I actually wrote this message yesterday but completed it with additional tests performed today. Sorry if I mixed yesterday and today in the text.

Sven Neumann
2004-12-05 03:57:42 UTC (over 19 years ago)

Some tests of GIMP startup time over NFS

Hi,

thanks for coming up with these numbers. IMO it's worthwhile to look at GIMP startup time and see if we can improve things there. I have an idea that might help in this regard and motivated by your mail, I think I should try to present it here...

Quite some time seems to be spent reading in data files like brushes, palettes, patterns, gradients, fonts. This gets worse if you start to add more than what comes with a default GIMP installation. So it seems that there's some potential for optimization here. Except for fonts which are handled somewhat differently, the data types I mentioned all all derived from a common base class, GimpData. So we should be able to improve things a lot if we can improve how GimpData loads it's file.

Or when. There is actually no good reason to load all brushes on startup, nor all palettes, patterns,... Actually it's more like a waste of resources to load them all into memory. People would probably install a lot more data files if that wouldn't make GIMP heavier and slow down startup.

But as soon as more data files are being added to GIMP, another problem starts to show up. The lack of categories makes it difficult to handle large amounts of brushes, palettes, patterns, gradients, fonts. At some point the list just becomes too long. Now this problem could be addressed at the same time...

So here's the proposal:

Similar to how plug-ins are cached in pluginrc, we could cache information about our data files. The brushrc would contain a list of all brushes. For each brush we would store name, size, filename and file modification time. Each data item also has a list of categories it belongs to. The GimpDataEditor can then use this info to group brushes into folders (see bug #119874). What's missing is a preview of the brush to use for example in the brush dialog.

Now if there was a PNG thumbnail for each data file, we could read that image instead. For large brushes and especially for animated brushes, this should be an improvement. For small brushes it would add an overhead though. If we have to read a 1000 thumbnail files instead of a 1000 brush files, that wouldn't improve things at all. However if we put all brush thumbnails into a single file, that should load rather fast. We could then even mmap the file at startup and delay reading the preview to the point where it is actually needed.

This would reduce data loading at startup to parsing a couple of rc files, attempting to mmap the preview archive or loading the file into memory if mmap fails. Only at first startup would we have to scan all brushes, palettes, patterns, ... GIMP would then write the rc files and generate a file with thumbnails. On subsequent starts GIMP would only check that the data files still match the cached versions.

If someone wants to work on this stuff, please speak up. I won't have time to spend on this feature anytime soon but I would try to help if somone's interested.

Sven