Hello,

Not being a GEGL or OpenGL developer, I would nevertheless being interested in how acceleration of filters etc. can basically work with OpenGL (with matrix operations on buffers? Can you do other operations, too?) because I always thought OpenGL was an API for accelerated (3D but also 2D, considering it as a plane in 3D space) rendering, but not for calculating (like it is required with many filters). Wouldn't it, from a technical view, be more useful to implement CUDA (http://www.nvidia.com/cuda/) support? I heard that there is a similar API for ATI. I also know that there is a GIMP plugin using CUDA (it has won the first place of a CUDA challenge: http://www.nvidia.co.uk/object/io_1222782056939.html).

As far as I have seen taking a quick look at the GEGL source code, the hardest task seems to be changing the single-threaded algorithm while (more_work) render();
to
for (i = 0; i < work_packets; i++) create_thread(render_packets);
where the packets are rectangles.

This would introduce a parallelism which would allow to 1) use all CPUs (would be very useful too, especially if you don't have 3D graphics acceleration available, but more than 1 CPU) 2) use CUDA to process the threads directly using GPU shaders (in parallel).

However, I have to say that I have no GEGL experience and little OpenGL and CUDA experience, so maybe I have some wrong assumptions, so feel free to correct me if I just wrote nonsense.

Hello,

Thanks for your answer.

As far as I am concerned, CUDA and ATI's Stream SDK aren't an option because they are video card specific. I haven't really looked into them but I'm sure they are easy to use for computations, but the same computations can be accomplished through OpenGL through multiple rendering/computation passes. Moreover, we decided to go with OpenGL instead of a cross-platform API like OpenCL because implementations for the latter are still either very young or non-existent for most major platforms[1].

Yes, I hope that all needed operations can be done via OpenGL, too because I have already experienced that CUDA sometimes makes problems even on supported platforms if you don't have exactly the same compiler options as the samples etc. On the other side, it's an additional unnecessary layer because the OpenGL computations use the same things you can directly access via CUDA or equivalent SDKs (if I have interpreted the infos I read about CUDA correctly). This layer is optimized for rendering and not for computing, so - from a pure simplicistic view - CUDA would be easier and more performant than OpenGL.

I have never heard of OpenCL, but it seems to be very interesting (and not proprietary). I hope that it will be available for the most important platforms, soon.

Sorry, but I'm not really familiar with CUDA's threading model.

Basically it's just the same as normal CPU threads, which would IMO be important to support, too. Many (especially Linux) systems don't have graphics acceleration available, but more than one CPU. Today you get already 8 cores, i.e. filters would be ~8 times faster if they utilised all CPUs (for at least 8 available rectangles).

Furthermore, GEGL doesn't really take care of thread creation and management.

I have seen this. But isn't that bad? My opionion is that it should.

As far as I can see, your code's intent is to divide the tasks into smaller packets of rectangle. GEGL already has GEGL processors that can be used to accomplish this[2]. In fact, internally, GEGL uses tiles to divide the image into smaller packets for on demand rendering and caching.

Yes I have found the relevant code sections in the GEGL source (at least I think that I have ;) ), and they seem to have the linear structure I have mentioned in my first email. I mean that gegl_processor_work() returns a boolean telling you if there is some more work to do, in which case you have to call it again. I think this is not usable for parallelism, because you always have to wait until the current packet (processed by gegl_processor_work()) is finished until you know if you have to call it again. For parallelism, a better approach would be that gegl_process_work() does everything at once, for instance by moving the outer while loop over the rectangles into in inner for loop.

I just wonder if that would help for the pure OpenGL approach, too. But I think that processing with OpenGL _and_ multithreading would make the best of pure OpenGL acceleration as well.

I will have a look at the articles on gpgpu.org, too, seems to be a very interesting site, thanks.

I am not at all sure that CUDA is an option at all or even OpenCL can be used. They are a more higher level api based implementations tweeked for computing. The thing for GEGL is quite a low level. You have to start from the extreme basic and the best way to start is using GLSL (makes more sence than HLSL or Cg). CUDA or OpenCL are all together different concepts and they dont let the user see any shaders or other implementations. It can easily be done using GLSL.

I applied for GSOC but had not got it but still I am very very interested it this project particular. I just wish to take part in it and just wish if you keep on posting on the mailing list.

On Sat, May 9, 2009 at 6:05 PM, Richard H. wrote:

Hello,

Thanks for your answer.

As far as I am concerned, CUDA and ATI's Stream SDK aren't an option because they are video card specific. I haven't really looked into them but I'm sure they are easy to use for computations, but the same computations can be accomplished through OpenGL through multiple rendering/computation passes. Moreover, we decided to go with OpenGL instead of a cross-platform API like OpenCL because implementations for the latter are still either very young or non-existent for most major platforms[1].

Yes, I hope that all needed operations can be done via OpenGL, too because I
have already experienced that CUDA sometimes makes problems even on supported
platforms if you don't have exactly the same compiler options as the samples
etc. On the other side, it's an additional unnecessary layer because the OpenGL computations use the same things you can directly access via CUDA or equivalent SDKs (if I have interpreted the infos I read about CUDA correctly).
This layer is optimized for rendering and not for computing, so - from a pure
simplicistic view - CUDA would be easier and more performant than OpenGL.

I have never heard of OpenCL, but it seems to be very interesting (and not proprietary). I hope that it will be available for the most important platforms, soon.

Sorry, but I'm not really familiar with CUDA's threading model.

Basically it's just the same as normal CPU threads, which would IMO be important to support, too. Many (especially Linux) systems don't have graphics
acceleration available, but more than one CPU. Today you get already 8 cores,
i.e. filters would be ~8 times faster if they utilised all CPUs (for at least
8 available rectangles).

Furthermore, GEGL doesn't really take care of thread creation and management.

I have seen this. But isn't that bad? My opionion is that it should.

As far as I can see, your code's intent is to divide the tasks into smaller packets of rectangle. GEGL already has GEGL processors that can be used to accomplish this[2]. In fact, internally, GEGL uses tiles to divide the image into smaller packets for on demand rendering and caching.

Yes I have found the relevant code sections in the GEGL source (at least I think that I have ;) ), and they seem to have the linear structure I have mentioned in my first email. I mean that gegl_processor_work() returns a boolean telling you if there is some more work to do, in which case you have
to call it again. I think this is not usable for parallelism, because you always have to wait until the current packet (processed by gegl_processor_work()) is finished until you know if you have to call it again. For parallelism, a better approach would be that gegl_process_work() does everything at once, for instance by moving the outer while loop over the
rectangles into in inner for loop.

I just wonder if that would help for the pure OpenGL approach, too. But I think that processing with OpenGL _and_ multithreading would make the best of
pure OpenGL acceleration as well.

I will have a look at the articles on gpgpu.org, too, seems to be a very interesting site, thanks.

--
Richard H. (via www.gimpusers.com)
_______________________________________________ Gegl-developer mailing list
Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer

I believe you are much off the track and you havent understood what you are supossed to do. It is not that we are forced to use OpenGl but it is that OpenGl is the best option. What you are telling is that OpenCl or Cuda can be used but that is purely wrong. Why you have to complicate procedures by using the OpenCL abstraction layer, of which there is no need as the things can be easily done using OpenGl. Infact I am sure you havent understood the whole of the problem at all. You feel for the support of the GPU since there is an API called OpenCL exists so you can user it. But it is not that way. You dont understand what OpenCL or CUDA is and I am sure you havent worked much upon it. Its a compute platform it provides API lot different that what is needed by GEGL. Infact as I previously said it is no way an option. I am sure you misunderstood the thing from the basics. You can create a minimap and use it to distribute the workload of the threads to either CPU using software rendering or Gpu using Hardware rendering or even to a network or something. I dont know why you are running out of the track.

Since you are Officially accepted for this project and I was Officially rejected so I expected you might had a well plan but you idea of using OpenCL or CUDA seems to me extremely Illogical. It would just not cause the increase the speed as can be got with basic implementation. This project is not a big and difficult task.

Really, my only concern with CUDA (and Stream SDK) is that they are too vendor-specific (I'd loved to be proven wrong though). So, implementing GPU-support through them is a no-go, afaic. I'm not willing to implement two GPU-enabled GEGL branches, one for NVIDIA and one for ATI video cards. That would be too much hassle and would not fit sensibly into GSoC's timeline of 3 months. I am willing to develop for GIMP and GEGL after GSoC but I would prefer my targets realistic. I really do agree that coding against CUDA and similar APIs is easier than coding against OpenGL, I'm just not convinced that the latter is an 'unnecessary' investment given our particular case. The only API that I would really approve of is OpenCL, but as I have previously said, OpenCL implementations are still in their infancy.

I really really feel that have you never been exposed to shaders or ever used GLSL or HLSL , because what I feel you dont have the understanding of the topic and it would really be too difficult for you to complete this. I am working upon GPU for last 2.5 years and upon things like cloud computing since last year.

On Sun, May 10, 2009 at 4:32 PM, Jerson Michael Perpetua < jersonperpetua@gmail.com> wrote:

Thanks for your answer.

No problem. :)

Yes, I hope that all needed operations can be done via OpenGL, too

because I

have already experienced that CUDA sometimes makes problems even on

supported

platforms if you don't have exactly the same compiler options as the

samples

etc. On the other side, it's an additional unnecessary layer because the OpenGL computations use the same things you can directly access via CUDA

or

equivalent SDKs (if I have interpreted the infos I read about CUDA

correctly).

This layer is optimized for rendering and not for computing, so - from a

pure

simplicistic view - CUDA would be easier and more performant than OpenGL.

I have never heard of OpenCL, but it seems to be very interesting (and

not

proprietary). I hope that it will be available for the most important platforms, soon.

Really, my only concern with CUDA (and Stream SDK) is that they are too vendor-specific (I'd loved to be proven wrong though). So, implementing GPU-support through them is a no-go, afaic. I'm not willing to implement two GPU-enabled GEGL branches, one for NVIDIA and one for ATI video cards. That would be too much hassle and would not fit sensibly into GSoC's timeline of 3 months. I am willing to develop for GIMP and GEGL after GSoC but I would prefer my targets realistic. I really do agree that coding against CUDA and similar APIs is easier than coding against OpenGL, I'm just not convinced that the latter is an 'unnecessary' investment given our particular case. The only API that I would really approve of is OpenCL, but as I have previously said, OpenCL implementations are still in their infancy.

Nevertheless, thank you for your concern. We are of the same spirit. I just hope that we have a mature, non-proprietary, vendor-neutral, platform-agnostic solution here and now. But I'm forced to use OpenGL as it is what is currently available, imho. But rest assured, I will commit myself into porting my OpenGL implementation into OpenCL as soon as implementations for the latter becomes available. I also hope that I'd have help by that time. :)

Basically it's just the same as normal CPU threads, which would IMO be important to support, too. Many (especially Linux) systems don't have

graphics

acceleration available, but more than one CPU. Today you get already 8

cores,

i.e. filters would be ~8 times faster if they utilised all CPUs (for at

least

8 available rectangles).

Implementing multi-threading in GEGL is out of my scope and I'm not even sure if it's in GEGL's scope. GEGL is pretty low-level and threading can be implemented on top of GEGL by the relevant client (i.e. GIMP). Furthermore, be aware that threading doesn't really map into multiple cores and not using threads doesn't really mean that the code will not be parallelized[1].

Currently, the changes that I propose to introduce GPU-support to GEGL will be added to GEGL's existing code paths so that when OpenGL and/or hardware acceleration isn't available, we will be able to fallback to plain software rendering (using the CPU).

Yes I have found the relevant code sections in the GEGL source (at least

I

think that I have ;) ), and they seem to have the linear structure I have mentioned in my first email. I mean that gegl_processor_work() returns a boolean telling you if there is some more work to do, in which case you

have

to call it again. I think this is not usable for parallelism, because you always have to wait until the current packet (processed by gegl_processor_work()) is finished until you know if you have to call it again. For parallelism, a better approach would be that

gegl_process_work()

does everything at once, for instance by moving the outer while loop over

the

rectangles into in inner for loop.

I'm not really an expert with regards to how GEGL uses threading frameworks (if at all) to parallelize tile/rectangle-level operations. My work will be concerned with accelerating pixel-level operations by parallelizing each pixel operation in the GPU. The key in my work is that all pixel operations should (theoretically) work in parallel. I'm not in the position to think about parallelizing tiles/rectangles though I suspect that parallelization in that area is limited because of GEGL's complex graph dependencies. I'd appreciate it if a GEGL expert steps up and clarify these issues.

[1] GCC, for example, will later provide automatic parallelization through the Graphite framework. Please see:

http://blog.internetnews.com/skerner/2009/04/gcc-44-improves-open-source-co.html and http://www.phoronix.com/scan.php?page=news_item&px=NzIyMQ. _______________________________________________ Gegl-developer mailing list
Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer

Sorry from my side if you felt something. I did not mean to hurt you. Obviously you must be better than me. Well by the minimap I mean you create a mapping of what part of buffer data is where, wether it is processed in CPU or in GPU or over the network. By this way You reduce the tranfer of data. Basically in GPU the costliest thing is the Data transfer than the processing. I dont feel such thing is used frequently but if you want to make a unified memory architechture then it would be much better. This thing will easily fit into the GEGL api.
I dint said OpenCL can not be used I just said OpenGl can do all the things needed and OpenCL or CUDA although can do this but they would be extra burden. OpenCl and CUDA use the Unified Compute architechture but I feel Gimp has much more worries of Data Transfer than the computation. CUDA or OpenCL is used best in places where large computation and less transfer is needed. I hope you understand. By this way of mapping you not only save the Data Tranfer but the architechture is much more usable even of you try Cloud Computing.

On Mon, May 11, 2009 at 4:03 PM, Jerson Michael Perpetua < jersonperpetua@gmail.com> wrote:

Hello,

Sorry that you feel that way. I did tell the GIMP and GEGL developer's during proposal period that I had no previous experience with OpenGL. Moreover, you can read from my mailing list posts that I admit to being a beginner in OpenGL, GObjects and GEGL. I can't really speculate why they accepted me against someone experienced like you. But one thing's for sure, I can and am willing to learn.

As to me being off-track, maybe we are approaching the same problem in different perspectives? What I've written in my GSoC proposal is a very murky way of parallelizing pixel operations by loading buffer image data into OpenGL textures and using those textures inside shaders. It's now extended into off-loading pixel data copy operations and conversions to the GPU, so that the CPU can save several cycles to do other tasks. In my view, OpenCL and similar APIs aren't very different with respect to how I am envisioning to use OpenGL for GEGL. I don't see why I can't use those so called computing languages. Sure, I'm not positive that all operations could be rewritten into shaders. What I'm sure is that the very basic of them I could implement in time for GSoC.

I'm curious about your mention of minimaps though. I haven't really heard about them being used in GPGPU techniques. The only thing I know about minimaps is that they are "miniature map[s], often placed in a corner of the screen in computer games to aid in reorientation" [from Wikipedia]. Maybe you can shed some light into this dark mind?

Respectfully yours, Daerd

Hello,

Implementing multi-threading in GEGL is out of my scope and I'm not even sure if it's in GEGL's scope.

I understand that as your project's task is adding OpenGL support. In my opinion, multi-CPU support would be more important, but that is of course another big task/project.

GEGL is pretty low-level and
threading can be implemented on top of GEGL by the relevant client (i.e. GIMP).

I don't understand this. How should GIMP provide threading support for GEGL processors? I mean, if you for instance scale a picture using GEGL, it is one GEGL operation. How can GIMP create multiple threads for this?

Furthermore, be aware that threading doesn't really map into multiple cores and not using threads doesn't really mean that the code will not be parallelized[1].

I had a look at Graphite/autopar and can say: Wow! Splitting loops to multiple threads automatically
[http://gcc.gnu.org/ml/gcc/2009-03/msg00239.html] is a good thing, but I think it's not true that threads don't map into multiple cores automatically. That's what threads (or processes, when threads are processes with same address space) are for.

I'm not really an expert with regards to how GEGL uses threading frameworks (if at all) to parallelize tile/rectangle-level operations.

I think it doesn't use threads at all and is designed for single-threaded use only... :/

My work will be concerned with accelerating pixel-level operations by parallelizing each pixel operation in the GPU. The key in my work is that all pixel operations should (theoretically) work in parallel.

That would indeed be very nice to have.

On Mon, May 11, 2009 at 7:49 PM, Richard H. wrote:

Hello,

Implementing multi-threading in GEGL is out of my scope and I'm not even sure if it's in GEGL's scope.

I understand that as your project's task is adding OpenGL support. In my opinion, multi-CPU support would be more important, but that is of course another big task/project.

GEGL has been designed to have an API that will allows using it without concern of whether it internally is single threaded (like its current incarnation), threaded, multi-process, multi-host or multi-process with parallel tasks driving GeglProcessors for different subregions, or perhaps even with the workload divided in a stream processing like manner.

GEGL is pretty low-level and
threading can be implemented on top of GEGL by the relevant client (i.e. GIMP).

Nope,. see above.

My work will be concerned with accelerating pixel-level operations by parallelizing each pixel operation in the GPU.  The key in my work is that all pixel operations should (theoretically) work in parallel.

That would indeed be very nice to have.

A GPU based backend should be possible to integrate with some of the above outlined possible approaches of dividing labour, ideally it would be possible to use GEGL and let GEGL itself figured out how to best make use computational resources available.

/Øyvind K.