A preview widget for plug-ins is one of these items that has been on the Gimp todo list for a long time.

In order to get some feedback from other developers we have compiled a list of requirements for a preview widget. The list is based on our own experiences and previous discussions on this mailing list. Our goal is a preview widget that could be used by most plug-ins.

We would like to hear your opinions on the following points: * Is the list complete?
* Are the requirements sufficiently clear? * Are there any unnecessary requirements?

We are currently working on the API and a prototype implementation. Our current implementation is based on Shawn Amundson's GimpPreview widget. It covers most of the requirements in this list, but is still mainly alpha code.

When there are no major additions to these requirements we hope to publish an API proposal in the next weeks.

greetings,

Ernst Lippe Maurits Rijk

Requirements for a GIMP plug-in preview widget ==============================================

This document gives a possible list of requirements for a preview widget that can be used by GIMP plug-ins. This is the version 1.0. This document was written by Maurits Rijk and Ernst Lippe .

Requirement 1. A plug-in author must be able to write a single version of the rendering function that can both be used for rendering to the plug-in and for rendering to the final drawable.

Far too many plug-ins that have a preview contain two different versions of the rendering algorithm: one for the preview and another one for the final result. For plug-ins that don't have a preview yet it is very desirable that the interface for generating a rendered image for the preview is very similar to the interface for generating the final results.

Requirement 2. The preview must support a GUI for scrolling through the image.

There are two possible GUI styles: dragging in the preview and using scrollbars. These could also be combined.

Open issue: Scrollbars make the widget visually bigger and makes its internal structure more complicated. The alternative of giving the preview widget two GtkAdjustments, that can be used by the developer to "wrap" the widget with scroll-bars, does not work when the preview widget has a visible scroll-bar and/or zoom controls. Scroll-bars will not be supported in the first release of the preview widget.

Requirement 3. When dragging can be used to scroll the preview it should show a "move" cursor in the preview image.

This gives visual feedback to the user that the image can be scrolled.

Requirement 4. During scrolling the preview should optionally show a (possibly scaled) version of the original image.

In many cases rendering algorithms are too slow to support real-time scrolling. It must be possible to turn this feature off. This would be better when the rendering algorithm is fast and when the rendered result bears little resemblance to the original image.

Requirement 5. The preview must support zooming.

Viewing a rendered image at different scales is very useful for a wide range of plug-ins.

Open issue: Should the preview accept arbitrary floating point numbers as scale factors or should it only accept a more limited set of different magnifications, e.g. of the form 1/n and n, where n is an integer. The latter approach can be more efficient in the implementation. It would even be more efficient when the preview only accepted only a limited set of magnifications, e.g 1/16, 1/15, ... 1/2, 1, 2, ... 16, because specialized code could be written for each magnification.

Requirement 6. The preview must contain an optional GUI for zooming.

A standard GUI for zooming the preview increases the uniformity of plug-ins and makes life easier for plug-in writers. It must be possible to hide the zooming GUI for previews that either don't support zooming or use a different interface.

Open issue: What should the GUI look like? A commonly used approach is to have a "+" and "-" button and a label to show the current scale. Another suggestion was to use spin-buttons. Because it seems most desirable that the scaling factors are more or less exponential the standard Gtk spinbuttons are not very useful. The first release of the preview widget will use "+" and "-" buttons.

Requirement 7. The preview must be able to handle both scaled and unscaled rendered data.

In some cases the rendering algorithm may be able to produce a scaled version of its outputs. In many cases the rendering algorithm cannot easily produce scaled data and then the preview should do the scaling.

Requirement 8. The scaling algorithm must be stable under scrolling.

The user must have the impression of scrolling through a fixed scaled version of the image. When the scaling algorithm is not stable, the preview will flicker during the scroll, which is highly annoying. In most cases this is caused by rounding errors. It is surprisingly difficult to write a good scaling algorithm due to these numerical problems.

Requirement 9. During zooming the preview should attempt to keep the center of the previewed image at the same position.

This is similar to the visual behaviour of zooming with a camera.

Requirement 10. The preview must emit a signal when the user has scrolled or zoomed the preview.

This signal can be used to synchronize multiple previews (e.g. see Filter Pack). This signal should contain information about the new position and/or scale. This signal will be emitted before the preview attempts to update the rendered image. The signal will only be emitted when the preview was scrolled by the user via the GUI. The signal will not be emitted when scrolling or zooming through the API.

Open issue: In principle the signal handler could be used to veto the scrolling or zooming. This will be included in the first release. Is this desirable?

Requirement 11. The preview should have an option to emit signals about the scrolling position while the user is still scrolling the image.

For efficiency reasons it is in general desirable that the scrolling signal is only emitted when the user has stopped scrolling. However, when the plug-in developer wants to synchronize scrolling multiple previews this signal must also be emitted during the scroll.

Requirement 12. The preview must support an API to scroll the preview and change the magnification.

This functionality is needed to synchronize multiple previews.

Requirement 13. The preview must be able to halt the rendering algorithm.

Frequently, the user will perform actions like scrolling or zooming that makes the image, that is being rendered by the rendering function, obsolete. There must be some way for the preview to inform the rendering function that it can stop.

Comment (el): I currently implemented this with an extension of the gimp_progress_bar. Any decent rendering function should already have some mechanism to indicate its progress. The gimp_progress_bar_update function returns false when the user has canceled the plug-in with the cancel button on the window that contains the original image, so it seems natural to use a similar approach for the preview.

Requirement 14. The user must be able to continue scrolling and zooming even when the rendering function is still rendering a new image.

With slow rendering algorithms it is intolerable when the widget "locks up" during rendering. When the user scrolls or zooms the preview, the image that is currently being rendered becomes obsolete.

Requirement 15. The preview must supply a mechanism that prevents showing obsolete rendered images.

It is possible that the rendering function will attempt to display image data in the preview that already has been obsoleted because the preview was scrolled or zoomed. The preview should ignore such obsolete drawing requests.

Requirement 16. The preview must have an optional progress bar to indicate the progress of the rendering function.

With slow rendering functions it is not obvious to the user when the rendering function has updated the display. It must be possible to hide the progress bar.

Requirement 17. The preview must be resizable at run-time.

In some cases users would like to enlarge the preview to get a better overview of the results. On the other hand large previews take more time to render and may clutter up the display. A resizable preview allows the user to select an optimal size. This requirement does not imply that the preview should offer a GUI for resizing, but only that it should respond when Gtk has allocated a new size for the preview.

Requirement 18. The preview must support incremental updates of rendered image data.

With slow rendering algorithms it may be desirable to update the preview as soon as new parts of the image have been rendered.

Comment (el): This requirement comes from the GIMP TODO list. So far I have not been able to implement this because Gtk only executes draws in an idle function with a low priority. So in my implementation the preview will only be redrawn when the rendering function has finished.

Requirement 19. The preview must handle alpha by showing a checkerboard pattern.

The GIMP GUI always handles alpha in this way.

Open issue: Must the checkerboarding pattern scroll with the image or remain static. The latter approach seems better with images that contain gray areas that are adjacent to transparent areas, but it is different from the conventions in other parts of the GIMP.

The following points have been mentioned in discussions about the preview widget. They are listed here because we explicitely don't want to support them.

Non-requirement 1. Preview the rendered results in the original image window.

The rendered image should be shown in the original image window.

Comment: This is one of the suggestions from the GUAD3C meeting. It does not seem relevant for the preview widget.

Non-requirement 2. Split preview window with before and after version of the image.

The user must be able to split the preview into two parts, one part showing the original image and the other part showing the rendered image.

Open issue: How should the preview be split? Suggestions have been left/right, top/bottom and even diagonally.

Comment: This is one of the suggestions from the GUAD3C meeting. It is not clear what the GUI for splitting/unsplitting should be. Also it seems much more useful to show before/after versions of the same area and not only for adjoining areas (which was the proposal, if I understood it correctly). Another disadvantage is that it makes the preview-code more complicated. When you really want to show both before and after it seems much easier to have two different previews that are synchronized with each other.

On 26 Feb 2003 14:24:17 +0100
Sven Neumann wrote:

Hi,

Ernst Lippe writes:

There are two possible GUI styles: dragging in the preview and using scrollbars. These could also be combined.

Open issue: Scrollbars make the widget visually bigger and makes its internal structure more complicated. The alternative of giving the preview widget two GtkAdjustments, that can be used by the developer to "wrap" the widget with scroll-bars, does not work when the preview widget has a visible scroll-bar and/or zoom controls. Scroll-bars will not be supported in the first release of the preview widget.

I don't understand the problem here. IMO using two adjustments to control the displayed area is very convenient and makes it perfectly easy to add scroll-bars. I'd suggest you try to come up with an API that uses adjustments. Perhaps you could outlines what problems you see here.

The point is the following. In the current implementation the preview widget consists of the following components from top to bottom: the image, the progress bar and the zoom controls. When scrollbars should be added the horizontal scrollbar should be located between the progress bar and the image. So it is not possible to add scrollbars by simply wrapping the entire current preview but the preview itself must be modified. Adding scrollbars makes the layout algorithm more complex.

I am not a great fan of using scrollbars for the preview. They make the widget a lot bigger and scrollbars are not easy to use with small previews.
When there really is an overwhelming demand for scrollbars, we will probably add them.

BTW, the most recent implementation uses adjustments for the position and the scale.

Open issue: What should the GUI look like? A commonly used approach is to have a "+" and "-" button and a label to show the current scale. Another suggestion was to use spin-buttons. Because it seems most desirable that the scaling factors are more or less exponential the standard Gtk spinbuttons are not very useful. The first release of the preview widget will use "+" and "-" buttons.

please consider to use the icons provided by GTK+:

http://developer.gnome.org/doc/API/2.0/gtk/gtk-Stock-Items.html#GTK-STOCK-ZOOM-IN-CAPS http://developer.gnome.org/doc/API/2.0/gtk/gtk-Stock-Items.html#GTK-STOCK-ZOOM-OUT-CAPS

OK.

Requirement 10. The preview must emit a signal when the user has scrolled or zoomed the preview.

This signal can be used to synchronize multiple previews (e.g. see Filter Pack). This signal should contain information about the new position and/or scale. This signal will be emitted before the preview attempts to update the rendered image. The signal will only be emitted when the preview was scrolled by the user via the GUI. The signal will not be emitted when scrolling or zooming through the API.

I'd suggest not to include information about the new position and/or scale in the signal but to provide a way to retrieve this information from the preview widget.

This is just my default multi-threading paranoia. When you have multiple variables that can be updated in a multi-threading environment, it is in general wise to use copies that are known to be consistent with one another. I don't really know how Gtk uses threading, so perhaps it is not useful to include the information.

Actually if you go for two adjustments and expose them in your API you don't need to deal with signals at all since it should be sufficient to connect to the "value_changed" and "changed" signals of the two adjustments.

The reason for a separate signal is that this makes it possible to distinguish between between modifications that are initiated by the user via the preview GUI and modifications that were initiated programmatically via the API. When this distinction is never important the requirement could be dropped.

Requirement 12. The preview must support an API to scroll the preview and change the magnification.

This functionality is needed to synchronize multiple previews.

and again you get this all for free if you go for two adjustments. Synchronizing two previews would boil down to synchronizing the preview's adjustements.

When you have an API call to change both coordinates at the same time, this makes it easier to avoid unnecessary refreshes, otherwise the widget might try to refresh itself between modification of the first and second coordinate.

greetings,

Ernst

On Wed, 26 Feb 2003 14:17:21 +0100 Øyvind Kolås wrote:

* Ernst Lippe [030226 14:04]:

Non-requirement 1. Preview the rendered results in the original image window.

The rendered image should be shown in the original image window.

Comment: This is one of the suggestions from the GUAD3C meeting. It does not seem relevant for the preview widget.

It is relevant in terms of providing a consistent user interface, the place in a plug-in's GUI it is natural to place a toggle that controls wether the canvas is updated with the preview or not. Is in relation to other preview controls.

I am not convinced that this toggle button really logically belongs to the preview widget.

The code neccesary to do this at the moment is quite complex, and involves creating temporary layers, messing with the undo state of gimp and such. The preview code already wants to be a "proxy" for rendering smaller previews using the same API calls as for already modifying the actual image, it thus seems like a natural place to add, or at least think about such functionality.

I don't see that there is much common code between the preview widget and the code for showing the result in the original canvas. In the ideal world both would be part of a standard Gimp preview library. But for the time being I don't have time to work on this.

greetings,

Ernst

On Thu, 27 Feb 2003 00:49:50 +0100 "Branko Collin" wrote:

Sometimes, a rendering algorithm is very slow.

I know this all too well.

A user should be able
to switch off the automatic rendering of a preview.

I don't think this is part of the preview widget. It calls the plug-in to do the rendering and that seems the proper place to make the decision about rendering a new image or not.

greetings,

Ernst

On 26 Feb 2003 17:29:37 +0100
Sven Neumann wrote:

Hi,

Ernst Lippe writes:

The point is the following. In the current implementation the preview widget consists of the following components from top to bottom: the image, the progress bar and the zoom controls. When scrollbars should be added the horizontal scrollbar should be located between the progress bar and the image. So it is not possible to add scrollbars by simply wrapping the entire current preview but the preview itself must be modified. Adding scrollbars makes the layout algorithm more complex.

IMO the preview widget should only be the preview, nothing else. If possible it should expose two adjustments so that you can easily add scrollbars. Progress-bar, zoom-control and scrollbars don't belong to the preview widget itself. They can be added by a composite widget.

That is already in my current design. There is a bare preview widget that can do scaling and scrolling but has no GUI for these operations. There is another composite widget that can include a progress bar and zoom buttons. Having a standard composite widget makes life easier for developers and gives more uniformity among the plug-ins.

Actually if you go for two adjustments and expose them in your API you don't need to deal with signals at all since it should be sufficient to connect to the "value_changed" and "changed" signals of the two adjustments.

The reason for a separate signal is that this makes it possible to distinguish between between modifications that are initiated by the user via the preview GUI and modifications that were initiated programmatically via the API. When this distinction is never important the requirement could be dropped.

Why should a widget behave differently if changed by the user or programmatically via the API? That sounds like a broken concept.

Why? This is simply a method to get a hook for trapping operations that the user has performed on the GUI of the widget.

A set of previews that you want to synchronize is an example of a constraint based system where you want to solve a set of constraints among multiple objects. The naive implementation of such a system is to let each object synchronize with all others when its value is changed. In general this is not a very good architecture: * It is expensive, you need at least n * (n - 1) synchronizations. * It frequently leads to oscillatory behaviour.

As an example where you could get funny behavior, take two previews that show the area around a certain point at different magnifications. Assume that the user scrolls in preview A. Now A will update the position of B. Because B is updated it will attempt to update A's position. In all implementations that I can think of there are choices for the scale factor such that the new position for A is different from the position that was set by the user. So A's position changes again and A will try to update B a second time. Eventually, this will probably stabilize, but when there are 3 previews with different magnifications there are probably cases where the oscillations never stabilize.

The standard solution for these problems is to have some central arbitrator that makes global decisions for all objects.

When you have a seperate signal for user operations this is a nice hook for such an arbitrator. It is of course possible to implement an arbitrator without these signals but its implementation seems a lot messier. Probably you would need some global arbitration flag and change the way that "value-changed" signals are handled based on the value of this flag. You would also have to be careful about subsequent operations by the user before the arbitration computations are finished.

Requirement 12. The preview must support an API to scroll the preview and change the magnification.

This functionality is needed to synchronize multiple previews.

and again you get this all for free if you go for two adjustments. Synchronizing two previews would boil down to synchronizing the preview's adjustements.

When you have an API call to change both coordinates at the same time, this makes it easier to avoid unnecessary refreshes, otherwise the widget might try to refresh itself between modification of the first and second coordinate.

you will have to delegate the actual refresh to idle functions anyway so that shouldn't be a problem.

I've worked quite a bit with Delphi that extensively uses its own variant of signals and I found there that is was very usefull to wait with sending signals when you were updating multiple variables. Anyhow, this is more a design than a requirements discussion.

BTW, I will leave the requirement as it stands, exposing adjustments to for scrolling and zooming is an API.

greetings,

Ernst

On 01 Mar 2003 17:39:56 +0100
Sven Neumann wrote:

Hi,

Ernst Lippe writes:

A set of previews that you want to synchronize is an example of a constraint based system where you want to solve a set of constraints among multiple objects. The naive implementation of such a system is to let each object synchronize with all others when its value is changed. In general this is not a very good architecture: * It is expensive, you need at least n * (n - 1) synchronizations. * It frequently leads to oscillatory behaviour.

As an example where you could get funny behavior, take two previews that show the area around a certain point at different magnifications. Assume that the user scrolls in preview A. Now A will update the position of B. Because B is updated it will attempt to update A's position. In all implementations that I can think of there are choices for the scale factor such that the new position for A is different from the position that was set by the user. So A's position changes again and A will try to update B a second time. Eventually, this will probably stabilize, but when there are 3 previews with different magnifications there are probably cases where the oscillations never stabilize.

The standard solution for these problems is to have some central arbitrator that makes global decisions for all objects.

When you have a seperate signal for user operations this is a nice hook for such an arbitrator. It is of course possible to implement an arbitrator without these signals but its implementation seems a lot messier. Probably you would need some global arbitration flag and change the way that "value-changed" signals are handled based on the value of this flag. You would also have to be careful about subsequent operations by the user before the arbitration computations are finished.

we usually solve this problem blocking the signal handlers when doing the update:

http://developer.gnome.org/doc/API/2.0/gobject/gobject-Signals.html#g-signal-handlers-block-by-func

This is IMO cleaner and simpler than adding an extra signal.

But blocking the signals on a GtkAdjustment only prevents the propagation of signals, it does not prevent an update of the underlying value. When the signal handlers are not sufficiently fast it is possible that the user has scrolled or zoomed while the signal handlers were blocked. In that case the underlying adjustment of the preview in which the user performed the operation will be updated but other components will not be notified. With scrolling this might be acceptable but with zooming the differences in magnification between multiple, supposedly synchronized, previews are very obvious. The only fundamental solution would be to block all updates by the user to each preview adjustment. I don't think that that this can be done on the adjustements themselves, the only solution I can see at the moment is to "freeze" all GUI components that could modify these adjustments.

greetings,

Ernst

On Sat, 01 Mar 2003 12:46:57 +0100 Michael Natterer wrote:

Ernst Lippe writes:

On 26 Feb 2003 17:29:37 +0100
Sven Neumann wrote:

Hi,

Ernst Lippe writes:

The point is the following. In the current implementation the preview widget consists of the following components from top to bottom: the image, the progress bar and the zoom controls. When scrollbars should be added the horizontal scrollbar should be located between the progress bar and the image. So it is not possible to add scrollbars by simply wrapping the entire current preview but the preview itself must be modified. Adding scrollbars makes the layout algorithm more complex.

IMO the preview widget should only be the preview, nothing else. If possible it should expose two adjustments so that you can easily add scrollbars. Progress-bar, zoom-control and scrollbars don't belong to the preview widget itself. They can be added by a composite widget.

That is already in my current design. There is a bare preview widget that can do scaling and scrolling but has no GUI for these operations. There is another composite widget that can include a progress bar and zoom buttons. Having a standard composite widget makes life easier for developers and gives more uniformity among the plug-ins.

Actually if you go for two adjustments and expose them in your API you don't need to deal with signals at all since it should be sufficient to connect to the "value_changed" and "changed" signals of the two adjustments.

The reason for a separate signal is that this makes it possible to distinguish between between modifications that are initiated by the user via the preview GUI and modifications that were initiated programmatically via the API. When this distinction is never important the requirement could be dropped.

Why should a widget behave differently if changed by the user or programmatically via the API? That sounds like a broken concept.

Why? This is simply a method to get a hook for trapping operations that the user has performed on the GUI of the widget.

A set of previews that you want to synchronize is an example of a constraint based system where you want to solve a set of constraints among multiple objects. The naive implementation of such a system is to let each object synchronize with all others when its value is changed. In general this is not a very good architecture: * It is expensive, you need at least n * (n - 1) synchronizations. * It frequently leads to oscillatory behaviour.

As an example where you could get funny behavior, take two previews that show the area around a certain point at different magnifications. Assume that the user scrolls in preview A. Now A will update the position of B. Because B is updated it will attempt to update A's position. In all implementations that I can think of there are choices for the scale factor such that the new position for A is different from the position that was set by the user. So A's position changes again and A will try to update B a second time. Eventually, this will probably stabilize, but when there are 3 previews with different magnifications there are probably cases where the oscillations never stabilize.

The standard solution for these problems is to have some central arbitrator that makes global decisions for all objects.

When you have a seperate signal for user operations this is a nice hook for such an arbitrator. It is of course possible to implement an arbitrator without these signals but its implementation seems a lot messier. Probably you would need some global arbitration flag and change the way that "value-changed" signals are handled based on the value of this flag. You would also have to be careful about subsequent operations by the user before the arbitration computations are finished.

You should make yourself familiar with what's actually possible when using GObject signals correctly.

The canonical way to solve the issue you mentioned is to let the widget itself perform it's operation in the signal's default handler (remember signals are just especially mighty virtual functions) and register the signal with G_SIGNAL_RUN_LAST. This way user signal connections are performed *before* the widget's default implementation and the user callback can choose to stop the signal or do whever voodoo it likes.

All the "hooks" and "central arbitrators" your are telking about are already implemented in the GSignal system.

BTW, the example of synchronizing two preview's scrolling offsets is rather an example of higher level logic than something the widgets themselves should provide. The "arbitrator" looks very much like a common signal handler that has some simple

g_signal_handlers_block()

/* do stuff */

g_signal_handlers_unblock()

stuff to sync the views. That's something implemented in the GIMP dozens of times and really nothing we have to worry about.

I believe you are proposing a similar solution as Sven. Please see my reply to his post, for the reason why I am not really comfortable with this proposal.

greetings,

Ernst

On 03 Mar 2003 11:54:42 +0100
Michael Natterer wrote:

Again, you should understand how GObject signals work. They operate purely synchronously. A signal handler cannot be "too slow" because the only thing your program is doing while emitting the signal is emitting the signal. When the call to g_signal_emit() returns all handlers have been called in order of connection.

But signal handlers can do anything they want, and so I assumed that they could also could also somehow invoke the main loop to process new events. I know at least one signal handler that works in this way. The rendering function for my preview indirectly calls the gtk main loop to process new events, because the user must be able to scroll and zoom the preview even while the rendering function is executing. But from your comments I guess that this is an exceptional case and that this is something that "good" signal handlers will not do.

The solution I propose is simply updating the preview in an idle function. This way you can change the underlying model as often as you like and the idle logic will make sure subsequent changes are compressed into a single update of the GUI.

Actually, the preview should update itself when it gets exposed. GTK already compresses and idles updates of the GUI, so everything that should be needed is calling gtk_widget_queue_draw() or gtk_widget_queue_draw_area() on the parts of the preview you want to update and the "expose" handler will do it's job of re-rendering at just the right time, namely immediately before the stuff is drawn to screen.

This seems a good idea. I have just rewritten my widget this way and it appears to be working fine.

greetings,

Ernst